How to implement Redis persistence

Redis is an advanced key-value database. It is similar to memcached, but data can be persisted and supports a wide range of data types. There are strings, linked lists, sets and sorted sets. It supports calculating the union, intersection and complement (difference) of sets on the server side, and also supports a variety of sorting functions.

Redis supports two persistence mechanisms: RDB and AOF. Persistence can avoid data loss caused by abnormal process exit or downtime, and can be used before the next restart. Persistent files enable data recovery.

RDB persistence

RDB persistence is a persistence method that saves complete data at a specific point in time by creating a compressed binary file snapshot. RDB persistence is the default persistence method of Redis. The triggering of RDB persistence includes manual triggering and automatic triggering.

Manually trigger

save. Execute the save command on the command line. The rdb file will be created synchronously to save the snapshot. This will block the main process of the server. Do not use bgsave in the production environment. In the command Executing the bgsave command will create an rdb file asynchronously and save the snapshot by forking a child process. Except for the blocking during fork, when the child process creates the rdb file, the main process can continue to process the request

Automatic trigger

Configure save m n in redis.conf to be triggered regularly. For example, save 900 1 means that when there is at least one update within 900s, master-slave replication is triggered. If the slave node performs a full replication operation, the master node automatically executes bgsave to generate RDB. file and sent to the slave node to execute the debug reload command. When reloading Redis, a shutdown is executed and AOF persistence is not enabled. RDB persistence configuration in redis.conf

# 只要满足下列条件之一，则会执行bgsave命令save 900 1 # 在900s内存在至少一次写操作save 300 10
save 60 10000# 禁用RBD持久化，可在最后加 save ""# 当备份进程出错时主进程是否停止写入操作stop-writes-on-bgsave-error yes# 是否压缩rdb文件 推荐no 相对于硬盘成本cpu资源更贵rdbcompression no

AOF persistence

AOF (Append- Only-File) persistence records all instructions that change the database status and appends them to the AOF file in the form of append. One way to rewrite it is: When the server is restarted, you can use the commands recorded in the AOF file to restore the database state when it was previously shut down.

The AOF persistence configuration in redis.conf is as follows

# 默认关闭AOF，若要开启将no改为yesappendonly no# append文件的名字appendfilename "appendonly.aof"# 每隔一秒将缓存区内容写入文件 默认开启的写入方式appendfsync everysec# 当AOF文件大小的增长率大于该配置项时自动开启重写（这里指超过原大小的100%）。auto-aof-rewrite-percentage 100# 当AOF文件大小大于该配置项时自动开启重写auto-aof-rewrite-min-size 64mb

The implementation of AOF persistence includes 3 steps:

Command append: append the command to the AOF buffer file to write Input: The buffer content is written to the AOF file. File save: The AOF file is saved to the disk. The frequency of the last two steps is configured through appendfsync. The options of appendfsync include

always, which is saved once every command is executed for security. The highest, only one command's data is lost at most, but the performance is also the lowest (frequent disk IO) everysec, saved once every second, recommended, a compromise between security and performance, no data lost for one second at most, depends on The operating system executes it (generally once every 30 seconds), with the lowest security and the highest performance. The data after the last time the operating system triggered the SAVE operation on the AOF file is lost. AOF is persisted through the save command. As time goes by, the AOF file will become larger and larger. Redis solves the problem of increasing AOF files by rewriting AOF files (which can reduce the file's disk occupancy and speed up data recovery). The principle is as follows:

Call fork to create A child process

The child process reads the status of the current database to "rewrite" a new AOF file (although it is called "rewriting" here, it does not actually read any old file, but Instructions are formed based on the current status of the database)

The main process continues to write new changes to the AOF rewrite buffer and the original AOF buffer at the same time

The main process obtains the child process rewrite Write the signal of AOF completion, call the signal processing function to write the contents of the AOF rewrite buffer into the new AOF file, rename the new file, atomically overwrite the original AOF file, and complete the replacement of the old and new files

AOF rewriting is also divided into manual triggering and automatic triggering

手动触发： 直接调用bgrewriteaof命令
自动触发： 根据auto-aof-rewrite-min-size和auto-aof-rewrite-percentage参数确定自动触发时机。其中auto-aof-rewrite-min-size表示运行AOF重写时文件最小体积，默认为64MB。auto-aof-rewrite-percentage表示当前AOF文件大小（aof_current_size）和上一次重写后AOF文件大小（aof_base_size）的比值。自动触发时机为 aof_current_size > auto-aof-rewrite-min-size &&（aof_current_size - aof_base_size）/aof_base_size> = auto-aof-rewrite-percentage

RDB vs AOF

RDB and AOF have their own advantages and disadvantages.

Advantages of RDB: Compared with AOF, RDB files are relatively smaller and data recovery is faster (see the data recovery section for reasons) Disadvantages of RDB: When the server is down, the RBD method will lose the last RDB persistence The final data; using bgsave to fork the child process will consume memory. Advantages of AOF: AOF only appends files, has less impact on server performance, is faster than RDB, consumes less memory, and has high readability. Using AOF will bring two disadvantages: the generated file is relatively large, even if it is rewritten by AOF, it will still be relatively large; at the same time, the speed of recovering data is also slower than RDB. Database recovery

If the AOF persistence function is not turned on, when the server starts, the main process will be blocked while the RDB file is automatically loaded. If the AOF persistence function is turned on, the server will give priority to using AOF files to restore the database state, because the update frequency of AOF files is usually higher than that of RDB files, and the saved data is more complete.

The processing flow of redis database recovery is as follows.

In terms of data recovery, the startup time of RDB will be shorter for two reasons:

In the RDB file, each There is only one record per data item, unlike the AOF log where a data item may have multiple operation records. So each piece of data only needs to be written once, and the file is relatively small.

RDB 文件的存储格式和Redis数据在内存中的编码格式是一致的，不需要再进行数据编码工作，所以在CPU消耗上要远小于AOF日志的加载。

但是在进行RDB持久化时，fork出来进行dump操作的子进程会占用与父进程一样的内存，采用的copy-on-write机制，对性能的影响和内存的消耗都是比较大的。比如16G内存，Redis已经使用了10G，这时save的话会再生成10G，变成20G，大于系统的16G。这时候会发生交换，要是虚拟内存不够则会崩溃，导致数据丢失。所以在用redis的时候一定对系统内存做好容量规划。

RDB、AOF混合持久化

Redis从4.0版开始支持RDB与AOF的混合持久化方案。首先由RDB定期完成内存快照的备份，然后再由AOF完成两次RDB之间的数据备份，由这两部分共同构成持久化文件。这个方案的优势在于其充分利用了RDB加载速度快、备份体积小以及AOF记录数据丢失几率尽可能低的特点。缺点是兼容性差，一旦开启了混合持久化，在4.0之前的版本都不识别该持久化文件，同时由于前部分是RDB格式，阅读性较低。

开启混合持久化

aof-use-rdb-preamble yes

数据恢复加载过程就是先按照RDB进行加载，然后把AOF命令追加写入。

持久化方案的建议 如果Redis只是用来做缓存服务器，比如数据库查询数据后缓存，那可以不用考虑持久化，因为缓存服务失效还能再从数据库获取恢复。建议同时采用两种持久化方式，以提高数据的安全性。如果你可以容忍在灾难发生时数据丢失几分钟，那么可以仅使用RDB。一般的设计方法是 使用主从复制机制以解决持久化时所带来的性能影响。即Master上RDB、AOF都不做，保证Master的读写性能，而Slave上则同时开启RDB和AOF（或4.0以上版本的混合持久化方式）来进行持久化，保证数据的安全性。

The above is the detailed content of How to implement Redis persistence. For more information, please follow other related articles on the PHP Chinese website!