在过去,查看redis的内存使用状态只有info memory命令,而且也只有一些基础信息,想要获取全局信息就有些困难。4.0开始redis提供了MEMORY
命令,一切都变得简单起来。
MEMORY
命令一共有5个子命令,可以通过MEMORY HELP
来查看:
127.0.0.1:6379> memory help
1) "MEMORY DOCTOR - Outputs memory problems report"
2) "MEMORY USAGE <key> [SAMPLES <count>] - Estimate memory usage of key"
3) "MEMORY STATS - Show memory usage details"
4) "MEMORY PURGE - Ask the allocator to release memory"
5) "MEMORY MALLOC-STATS - Show allocator internal stats"
接下来我们从MEMORY STATS
开始,一一介绍各个子命令的功能。
首先,我们需要明确一个概念,redis的内存使用不仅包含所有的key-value数据,还有描述这些key-value的元信息,以及许多管理功能的消耗,比如持久化、主从复制,通过MEMORY STATS
可以更好的了解到redis的内存使用状况。
这里我们启动了一个打开持久化功能并且带slave的redis,向其中随机写入了一些数据(某些数据还带有过期时间),以便读者可以更好的了解redis的内存使用,接下来执行MEMORY STATS
命令:
127.0.0.1:6379> memory stats
1) "peak.allocated"
2) (integer) 423995952
3) "total.allocated"
4) (integer) 11130320
5) "startup.allocated"
6) (integer) 9942928
7) "replication.backlog"
8) (integer) 1048576
9) "clients.slaves"
10) (integer) 16858
11) "clients.normal"
12) (integer) 49630
13) "aof.buffer"
14) (integer) 3253
15) "db.0"
16) 1) "overhead.hashtable.main"
2) (integer) 5808
3) "overhead.hashtable.expires"
4) (integer) 104
17) "overhead.total"
18) (integer) 11063904
19) "keys.count"
20) (integer) 94
21) "keys.bytes-per-key"
22) (integer) 12631
23) "dataset.bytes"
24) (integer) 66416
25) "dataset.percentage"
26) "5.5934348106384277"
27) "peak.percentage"
28) "2.6251003742218018"
29) "fragmentation"
30) "1.1039986610412598"
一共有15项内容,内存使用量均以字节为单位,我们一个一个来看:
主从复制中所有slave的读写缓冲区,包括output-buffer(也即输出缓冲区)使用的内存和querybuf(也即输入缓冲区),这里简单介绍一下主从复制:
此项为aof持久化使用的缓存和aofrewrite时产生的缓存之和,当然如果关闭了appendonly那这项就一直为0:
可以看出这一项的大小与写入流量成正比。
redis每个db的元信息使用的内存,这里只使用了db0,所以只打印了db0的内存使用状态,当使用其他db时也会有相应的信息。
db的元信息有以下三项:
a) redis的db就是一张hash表,首先就是这张hash表使用的内存(redis使用链式hash,hash表中存放所有链表的头指针);
b) 每一个key-value对都有一个dictEntry
来记录他们的关系,元信息便包含该db中所有dictEntry
使用的内存;
c) redis使用redisObject
来描述value所对应的不同数据类型(string、list、hash、set、zset),那么redisObject
占用的空间也计算在元信息中。
overhead.hashtable.main:
db的元信息也即是以上三项之和,计算公式为:
hashtable + dictEntry
+ redisObject
overhead.hashtable.expires:
对于key的过期时间,redis并没有把它和value放在一起,而是单独用一个hashtable来存储,但是expires这张hash表记录的是key-expire信息,所以不需要`redisObject`来描述value,其元信息也就少了一项,计算公式为:
hashtable + dictEntry
100 * dataset.bytes / (total.allocated - startup.allocated)
平均每个key的内存大小,直觉上应该是用dataset.bytes除以keys.count即可,但是redis并没有这么做,而是把管理类内存也平摊到了每个key的内存使用中,计算公式为:
(total.allocated - startup.allocated) / keys.count
相信所有redis用户都希望对每一个key-value的内存使用了如指掌,然而4.0之前redis并没有提供一个明确的方法来进行内存评估,不过从4.0开始,MEMORY命令实现了这一功能。
首先看下使用方法:MEMORY usage [samples]
命令参数不多,通过字面意思也可以看出来是评估指定key的内存使用情况。samples是可选参数默认为5,以hash为例看下其如果工作:
dictEntry
的内存占用信息。dictEntry
中key-value都是字符串,所以没redisObject
的额外消耗。在评估真正的数据内存大小时redis并没有去遍历所有key,而是采用的抽样估算:随机抽取samples个key-value对计算其平均内存占用,再乘以key-value对的个数即得到结果。试想一下如果要精确计算内存占用,那么就需要遍历所有的元素,当元素很多时就是使redis阻塞,所以请合理设置samples的大小。其他数据结构的计算方式类似于hash,此处就不再赘述。
此项子命令是作者给出的关于redis内存使用方面的建议,在不同的允许状态下会有不同的分析结果:
运行状态良好:
Hi Sam, I can't find any memory issue in your instance. I can only account for what occurs on this base.
redis的数据量很小,暂无建议:
Hi Sam, this instance is empty or is using very little memory, my issues detector can't be used in these conditions. Please, leave for your mission on Earth and fill it with some data. The new Sam and I will be back to our programming as soon as I finished rebooting.
内存使用峰值1.5倍于目前内存使用量,此时内存碎片率可能会比较高,需要注意:
Peak memory: In the past this instance used more than 150% the memory that is currently using. The allocator is normally not able to release memory after a peak, so you can expect to see a big fragmentation ratio, however this is actually harmless and is only due to the memory peak, and if the Redis instance Resident Set Size (RSS) is currently bigger than expected, the memory will be used as soon as you fill the Redis instance with more data. If the memory peak was only occasional and you want to try to reclaim memory, please try the MEMORY PURGE command, otherwise the only other option is to shutdown and restart the instance.
内存碎片率过高超过1.4,需要注意:
High fragmentation: This instance has a memory fragmentation greater than 1.4 (this means that the Resident Set Size of the Redis process is much larger than the sum of the logical allocations Redis performed). This problem is usually due either to a large peak memory (check if there is a peak memory entry above in the report) or may result from a workload that causes the allocator to fragment memory a lot. If the problem is a large peak memory, then there is no issue. Otherwise, make sure you are using the Jemalloc allocator and not the default libc malloc.
每个slave缓冲区的平均内存超过10MB,原因可能是master写入流量过高,也有可能是主从同步的网络带宽不足或者slave处理较慢:
Big slave buffers: The slave output buffers in this instance are greater than 10MB for each slave (on average). This likely means that there is some slave instance that is struggling receiving data, either because it is too slow or because of networking issues. As a result, data piles on the master output buffers. Please try to identify what slave is not receiving data correctly and why. You can use the INFO output in order to check the slaves delays and the CLIENT LIST command to check the output buffers of each slave.
普通客户端缓冲区的平均内存超过200KB,原因可能是pipeline使用不当或者Pub/Sub客户端处理消息不及时导致:
Big client buffers: The clients output buffers in this instance are greater than 200K per client (on average). This may result from different causes, like Pub/Sub clients subscribed to channels bot not receiving data fast enough, so that data piles on the Redis instance output buffer, or clients sending commands with large replies or very large sequences of commands in the same pipeline. Please use the CLIENT LIST command in order to investigate the issue if it causes problems in your instance, or to understand better why certain clients are using a big amount of memory.
打印内存分配器状态,只在使用jemalloc时有用。
请求分配器释放内存,同样只对jemalloc生效。
基于4.0的云redis已正式上线,登陆阿里云控制台即可开通:https://www.aliyun.com/product/kvstore