Detailed explanation of Redis RDB and AOF

Redis has two persistence solutions, RDB (Redis DataBase) and AOF (Append Only File). If you want to quickly understand and use RDB and AOF, you can jump directly to the bottom of the article to read the summary. This chapter uses configuration files, ways to trigger snapshots, data recovery operations, command operation demonstrations, advantages and disadvantages to learn the key knowledge persistence of Redis. This article mainly provides a detailed analysis of the two persistence solutions of Redis, RDB and AOF. We hope that the content we have compiled can help everyone have a deeper understanding of these two solutions.

RDB detailed explanation

RDB is the default persistence solution for Redis. Within the specified time interval, if a specified number of write operations are performed, the data in the memory will be written to the disk. That is, a dump.rdb file is generated in the specified directory. Restarting Redis will restore data by loading the dump.rdb file.

Understand RDB from the configuration file

Open the redis.conf file and find the corresponding content of SNAPSHOTTING
1 RDB core rule configuration (key points)

save <seconds> <changes>
# save ""
save 900 1
save 300 10
save 60 10000

Explanation: save a9c955145cca9183e472b3dab6d282ab cf423e1860ac2eff776feebdae61c479, synchronize the data in the memory to the hard disk when the conditions are met. The default official factory configuration is that if there is 1 change within 900 seconds, 10 changes within 300 seconds, and 10,000 changes within 60 seconds, the data snapshot in the memory will be written to the disk.

If you don’t want to use the RDB solution, you can open the save "" comment, the following three comments.

2 Specify the local database file name, usually the default dump.rdb

dbfilename dump.rdb

3 Specify the local database storage directory, usually Also use the default configuration

dir ./

4 Turn on data compression by default

rdbcompression yes

Explanation: Configuration storage Whether to compress the data when going to the local database, the default is yes. Redis uses LZF compression, but it takes up a little CPU time. If this option is turned off, the database file will become huge. It is recommended to turn it on.

Trigger RDB snapshot

1 Perform the specified number of write operations within the specified time interval

2 Execute save (blocking, Just save the snapshot and wait for the rest) or use the bgsave (asynchronous) command

3 to execute the flushall command to clear all data in the database, which is of little significance.

4 Execute the shutdown command to ensure that the server is shut down normally without losing any data, which is of little significance.

Restore data through RDB files

Copy the dump.rdb file to the bin directory of the redis installation directory, and restart the redis service. In actual development, the physical machine hard disk damage is generally considered and backup dump.rdb is selected. You can experience it from the following operation demonstration.

Advantages and Disadvantages of RDB

Advantages:

1 Suitable for large-scale data recovery.

2 If the business does not have high requirements for data integrity and consistency, RDB is a good choice.

Disadvantages:

1 The integrity and consistency of the data are not high because the RDB may have been down during the last backup.

2 Occupies memory during backup, because Redis will independently create a sub-process during backup and write data to a temporary file (at this time, the data in memory is twice the original size), Finally, replace the previous backup file with the temporary file.

Therefore, it is more reasonable to choose to execute Redis persistence and data recovery in the dead of night.

Operation Demonstration

[root@itdragon bin]# vim redis.conf
save 900 1
save 120 5
save 60 10000
[root@itdragon bin]# ./redis-server redis.conf
[root@itdragon bin]# ./redis-cli -h 127.0.0.1 -p 6379
127.0.0.1:6379> keys *
(empty list or set)
127.0.0.1:6379> set key1 value1
OK
127.0.0.1:6379> set key2 value2
OK
127.0.0.1:6379> set key3 value3
OK
127.0.0.1:6379> set key4 value4
OK
127.0.0.1:6379> set key5 value5
OK
127.0.0.1:6379> set key6 value6
OK
127.0.0.1:6379> SHUTDOWN
not connected> QUIT
[root@itdragon bin]# cp dump.rdb dump_bk.rdb
[root@itdragon bin]# ./redis-server redis.conf
[root@itdragon bin]# ./redis-cli -h 127.0.0.1 -p 6379
127.0.0.1:6379> FLUSHALL 
OK
127.0.0.1:6379> keys *
(empty list or set)
127.0.0.1:6379> SHUTDOWN
not connected> QUIT
[root@itdragon bin]# cp dump_bk.rdb dump.rdb
cp: overwrite `dump.rdb'? y
[root@itdragon bin]# ./redis-server redis.conf
[root@itdragon bin]# ./redis-cli -h 127.0.0.1 -p 6379
127.0.0.1:6379> keys *
1) "key5"
2) "key1"
3) "key3"
4) "key4"
5) "key6"
6) "key2"

Step 1: vim Modify the persistence configuration time. If it is modified 5 times within 120 seconds, it will be persisted. once.

Step 2: Restart the service to make the configuration take effect.

Step 3: Set 5 keys respectively. After two minutes, a dump.rdb file will be automatically generated in the current directory of bin. (set key6 is to verify that shutdown can trigger RDB snapshot)

Step 4: Make a backup copy of the current dump.rdb (simulating online work).

Step 5: Execute the FLUSHALL command to clear the database data (simulate data loss).

Step 6: Restart the Redis service and restore data... Huh? ? ? ? (′◔ ‸◔`). The data is empty? ? ? ? This is because FLUSHALL also has the function of triggering RDB snapshots.

Step 7: Replace the backup dump_bk.rdb with dump.rdb and then redis.

Note: SHUTDOWN and FLUSHALL commands will trigger RDB snapshots. This is a pitfall. Please pay attention.

Other commands:

keys * Match all keys in the database save Block trigger RDB snapshot to back up data FLUSHALL Clear the data of the entire Redis server (rarely used) SHUTDOWN Shut down and leave (rarely used) Use)

AOF Detailed explanation

AOF: Redis is not enabled by default. It appears to make up for the shortcomings of RDB (data inconsistency), so it uses a log to record each write operation and append it to the file. When Redis is restarted, the write instructions will be executed from front to back based on the contents of the log file to complete the data recovery work.

Understand AOF from the configuration file

Open the redis.conf file and find the corresponding content of APPEND ONLY MODE
1 redis is closed by default. To enable it, you need to manually change no to yes

appendonly yes

2 指定本地数据库文件名，默认值为 appendonly.aof

appendfilename "appendonly.aof"

 

3 指定更新日志条件

# appendfsync always
appendfsync everysec
# appendfsync no

 

解说：

always：同步持久化，每次发生数据变化会立刻写入到磁盘中。性能较差当数据完整性比较好（慢，安全）

everysec：出厂默认推荐，每秒异步记录一次（默认值）

no：不同步

4 配置重写触发机制

auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

 

解说：当AOF文件大小是上次rewrite后大小的一倍且文件大于64M时触发。一般都设置为3G，64M太小了。

触发AOF快照

根据配置文件触发，可以是每次执行触发，可以是每秒触发，可以不同步。

根据AOF文件恢复数据

正常情况下，将appendonly.aof 文件拷贝到redis的安装目录的bin目录下，重启redis服务即可。但在实际开发中，可能因为某些原因导致appendonly.aof 文件格式异常，从而导致数据还原失败，可以通过命令redis-check-aof --fix appendonly.aof 进行修复 。从下面的操作演示中体会。

AOF的重写机制

前面也说到了，AOF的工作原理是将写操作追加到文件中，文件的冗余内容会越来越多。所以聪明的 Redis 新增了重写机制。当AOF文件的大小超过所设定的阈值时，Redis就会对AOF文件的内容压缩。

重写的原理：Redis 会fork出一条新进程，读取内存中的数据，并重新写到一个临时文件中。并没有读取旧文件（你都那么大了，我还去读你？？？ o(ﾟДﾟ)っ傻啊！）。最后替换旧的aof文件。

触发机制：当AOF文件大小是上次rewrite后大小的一倍且文件大于64M时触发。这里的“一倍”和“64M” 可以通过配置文件修改。

AOF 的优缺点

优点：数据的完整性和一致性更高

缺点：因为AOF记录的内容多，文件会越来越大，数据恢复也会越来越慢。

操作演示

[root@itdragon bin]# vim appendonly.aof
appendonly yes
[root@itdragon bin]# ./redis-server redis.conf
[root@itdragon bin]# ./redis-cli -h 127.0.0.1 -p 6379
127.0.0.1:6379> keys *
(empty list or set)
127.0.0.1:6379> set keyAOf valueAof
OK
127.0.0.1:6379> FLUSHALL 
OK
127.0.0.1:6379> SHUTDOWN
not connected> QUIT
[root@itdragon bin]# ./redis-server redis.conf
[root@itdragon bin]# ./redis-cli -h 127.0.0.1 -p 6379
127.0.0.1:6379> keys *
1) "keyAOf"
127.0.0.1:6379> SHUTDOWN
not connected> QUIT
[root@itdragon bin]# vim appendonly.aof
fjewofjwojfoewifjowejfwf
[root@itdragon bin]# ./redis-server redis.conf
[root@itdragon bin]# ./redis-cli -h 127.0.0.1 -p 6379
Could not connect to Redis at 127.0.0.1:6379: Connection refused
not connected> QUIT
[root@itdragon bin]# redis-check-aof --fix appendonly.aof 
'x    3e: Expected prefix '*', got: '
AOF analyzed: size=92, ok_up_to=62, diff=30
This will shrink the AOF from 92 bytes, with 30 bytes, to 62 bytes
Continue? [y/N]: y
Successfully truncated AOF
[root@itdragon bin]# ./redis-server redis.conf
[root@itdragon bin]# ./redis-cli -h 127.0.0.1 -p 6379
127.0.0.1:6379> keys *
1) "keyAOf"

第一步：修改配置文件，开启AOF持久化配置。

第二步：重启Redis服务，并进入Redis 自带的客户端中。

第三步：保存值，然后模拟数据丢失，关闭Redis服务。

第四步：重启服务，发现数据恢复了。（额外提一点：有教程显示FLUSHALL 命令会被写入AOF文件中，导致数据恢复失败。我安装的是redis-4.0.2没有遇到这个问题）。

第五步：修改appendonly.aof，模拟文件异常情况。

第六步：重启 Redis 服务失败。这同时也说明了，RDB和AOF可以同时存在，且优先加载AOF文件。

第七步：校验appendonly.aof 文件。重启Redis 服务后正常。

补充点：aof 的校验是通过 redis-check-aof 文件，那么rdb 的校验是不是可以通过 redis-check-rdb 文件呢？？？

总结 Redis 默认开启RDB持久化方式，在指定的时间间隔内，执行指定次数的写操作，则将内存中的数据写入到磁盘中。 RDB 持久化适合大规模的数据恢复但它的数据一致性和完整性较差。 Redis 需要手动开启AOF持久化方式，默认是每秒将写操作日志追加到AOF文件中。

AOF 的数据完整性比RDB高，但记录内容多了，会影响数据恢复的效率。 Redis 针对 AOF文件大的问题，提供重写的瘦身机制。若只打算用Redis 做缓存，可以关闭持久化。若打算使用Redis 的持久化。建议RDB和AOF都开启。其实RDB更适合做数据的备份，留一后手。AOF出问题了，还有RDB。

相关推荐：

深入剖析 redis RDB 持久化策略

Redis的持久化--RDB的工作原理及引发的问题

深入剖析 redis AOF 持久化策略

The above is the detailed content of Detailed explanation of Redis RDB and AOF. For more information, please follow other related articles on the PHP Chinese website!