Methods and principles of Redis persistence snapshots

The so-called persistence is to keep our data from being lost, and the data is usually saved on our hard disk. There are two persistence methods in Redis, one is snapshot persistence and the other is AOF persistence. Each has its own advantages and disadvantages. In the project, we have to choose the specific persistence method according to the actual situation.

Recommended: redis introductory tutorial

Snapshot persistence (RDB)

It is also called RDB persistence method, which is through Persistence is achieved by taking snapshots, storing memory data at a certain time in an rdb file, and loading the data in the file when the redis service is restarted

Configuring persistent snapshots

Snapshot persistence in redis is enabled by default, and there are related configuration options in the redis.conf configuration file

################################ SNAPSHOTTING  ################################
#
# Save the DB on disk:
#
#   save <seconds> <changes>
#
#   Will save the DB if both the given number of seconds and the given
#   number of write operations against the DB occurred.
#
#   In the example below the behaviour will be to save:
#   after 900 sec (15 min) if at least 1 key changed
#   after 300 sec (5 min) if at least 10 keys changed
#   after 60 sec if at least 10000 keys changed
#
#   Note: you can disable saving completely by commenting out all "save" lines.
#
#   It is also possible to remove all the previously configured save
#   points by adding a save directive with a single empty string argument
#   like in the following example:
#
#   save ""

save 900 1   #900秒内至少有1个key被更改就执行快照
save 300 10  #300内描述至少有10个key被更改就执行快照
save 60 10000  #60秒内至少有10000个key被更改就执行快照

# By default Redis will stop accepting writes if RDB snapshots are enabled
# (at least one save point) and the latest background save failed.
# This will make the user aware (in a hard way) that data is not persisting
# on disk properly, otherwise chances are that no one will notice and some
# disaster will happen.
#
# If the background saving process will start working again Redis will
# automatically allow writes again.
#
# However if you have setup your proper monitoring of the Redis server
# and persistence, you may want to disable this feature so that Redis will
# continue to work as usual even if there are problems with disk,
# permissions, and so forth.
stop-writes-on-bgsave-error yes #拍摄快照失败是否继续执行写命令

# Compress string objects using LZF when dump .rdb databases?
# For default that&#39;s set to &#39;yes&#39; as it&#39;s almost always a win.
# If you want to save some CPU in the saving child set it to &#39;no&#39; but
# the dataset will likely be bigger if you have compressible values or keys.
rdbcompression yes #是否对快照文件进行压缩

# Since version 5 of RDB a CRC64 checksum is placed at the end of the file.
# This makes the format more resistant to corruption but there is a performance
# hit to pay (around 10%) when saving and loading RDB files, so you can disable it
# for maximum performances.
#
# RDB files created with checksum disabled have a checksum of zero that will
# tell the loading code to skip the check.
rdbchecksum yes #是否进行数据校验

# The filename where to dump the DB
dbfilename dump.rdb #快照文件存储的名称
# The working directory.
#
# The DB will be written inside this directory, with the filename specified
# above using the &#39;dbfilename&#39; configuration directive.
#
# The Append Only File will also be created inside this directory.
#
# Note that you must specify a directory here, not a file name.
dir /opt/redis-5.0.3#快照文件存储的位置

Verify the effect

1. Enter the installation directory and delete the dump.db file if there is one.

2. Start redis, then add a few data, then close redis and exit

[root@root redis-5.0.3]# src/redis-cli 
127.0.0.1:6379> ping
PONG
127.0.0.1:6379> set name aaa
OK
127.0.0.1:6379> set age 18
OK
127.0.0.1:6379> incr age
(integer) 19
127.0.0.1:6379> shutdown
not connected> exit

3. Generate a dump.rdb file in our installation directory. This is our snapshot backup file

4. Start redis again and enter to discover the original The data is still there because redis loads the data in the backup file when it starts.

[root@root redis-5.0.3]# src/redis-server redis.conf 
1211:C 13 Feb 2019 01:27:22.668 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1211:C 13 Feb 2019 01:27:22.668 # Redis version=5.0.3, bits=64, commit=00000000, modified=0, pid=1211, just started
1211:C 13 Feb 2019 01:27:22.668 # Configuration loaded
[root@root redis-5.0.3]# src/redis-cli 
127.0.0.1:6379> ping 
PONG
127.0.0.1:6379> get name
"aaa"
127.0.0.1:6379> get age
"19"
127.0.0.1:6379> keys *
1) "name"
2) "age"

5. Close and exit

Delete the dump.rdb file after closing and exiting. After starting, it is found that the data is gone

[root@root redis-5.0.3]# src/redis-server redis.conf 
1218:C 13 Feb 2019 01:29:01.336 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1218:C 13 Feb 2019 01:29:01.336 # Redis version=5.0.3, bits=64, commit=00000000, modified=0, pid=1218, just started
1218:C 13 Feb 2019 01:29:01.336 # Configuration loaded
[root@root redis-5.0.3]# src/redis-cli 
127.0.0.1:6379> ping 
PONG
127.0.0.1:6379> keys *
(empty list or set)

Snapshot persistence principle

save command:

While redis is running, we can explicitly send a save command to take a snapshot. The save command is a blocking command, that is, when the server receives a save command, it will start taking snapshots. During this period, it will not process other requests. Other requests will be suspended until the backup is completed

bgsave command

The bgsave command also takes a snapshot immediately. Different from the save command, bgsave is not a blocking command, but a fork sub-thread, and then this sub-thread is responsible for the backup operation. . The parent process continues to process the client's request, so that it will not cause blocking.

127.0.0.1:6379> ping
PONG
127.0.0.1:6379> keys *
(empty list or set)
127.0.0.1:6379> set name zhangsan
OK
127.0.0.1:6379> set age 20
OK
127.0.0.1:6379> bgsave
Background saving started

4. Shutdown command

When we only want to shutdown the command. The server will automatically send a save command to complete the snapshot operation. And shut down the server after completing the backup operation. Therefore, when our operations do not meet the previous three conditions, after we shut down the server, our data will not be lost when we open it again.

5. sync command

When in a master-slave environment, the slave node will send a sync command to develop a replication when it wants to synchronize the data of the master node. At this time, the master node will send a bgsave command to fork a new thread to complete the snapshot and send the snapshot file to the slave node after the bgsave command operation is completed to complete the data synchronization of the master and slave nodes.

Advantages and Disadvantages

Advantages

RDB files save redis data at a certain point in time and are suitable for backup. You can set a point in time to RDB files are archived so that the data can be easily restored to a different version when needed. RDB is very suitable for disaster recovery. Single files can be easily transferred to remote servers. When the amount of data is relatively large, RDB starts quickly.

Disadvantages

RDB can easily cause data loss. If it is set to be saved every 3 minutes, if redis cannot work properly for some reasons, then The data between the last snapshot and the problem with Redis will be lost.

How to disable snapshot persistence

1. Comment out all save configurations in the redis.conf configuration file 2. Add the eat command to the last save configuration

save ""

AOF persistence

Unlike snapshot persistence, which directly saves the key-value pair data of Redis, AOF persistence records the memory data of Redis by saving the write commands executed by Redis. Theoretically, as long as we save all commands that may modify Redis memory data (that is, write commands), then based on these saved write commands, we can restore the memory state of Redis. AOF persistence uses this principle to achieve data persistence and data recovery

Open AOF

In redis, aof is closed by default, we need to modify the configuration file to enable aof

appendonly yes
appendfilename "appendonly.aof"
# If unsure, use "everysec".
# appendfsync always
appendfsync everysec
# appendfsync no
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

Turn off snapshot persistence

save ""
#save 900 1
#save 300 10
#save 60 10000

Verify and restart the service

When executing a simple command operation, we can see that the content stored in the append.aof file is the command we executed

Notes on AOF persistent backup

1.appendfsync has three values, namely everysec, always and no. Here we recommend using everysec , always is not recommended. Because always will seriously affect the performance of the server. 2. In the worst case, everysec will only lose 1 second of data, and the impact is within controllable range.

Advantages and Disadvantages

Advantages

The AOF persistence method provides a variety of synchronization frequencies. Even if the default synchronization frequency is used to synchronize once per second, Redis will be lost at most. Just 1 second of data. The format of AOF files is highly readable, which also provides users with a more flexible processing method. For example, if we accidentally use the FLUSHALL command by mistake, before rewriting is in progress, we can manually remove the last FLUSHALL command and then use AOF to recover the data.

Disadvantages

Although AOF provides a variety of synchronization frequencies, by default, the frequency of synchronization once per second also has higher performance. But when the load on Redis is high, RDB has better performance guarantee than AOF. RDB uses snapshots to persist the entire Redis data, while AOF only appends each executed command to the AOF file, so in theory, RDB is more robust than the AOF method

Persistence Some usage suggestions

1. If redis is only used as a cache server, we can not use any persistence.

2. Under normal circumstances, we will enable both persistence methods. redis loads AOF files first to reply data. The advantage of RDB is that it is fast.

3. In the master-slave node, RDB serves as our backup data and is only started on the salve (slave node). The synchronization time can be set longer, leaving only (save 900 1) this rule. That's it.

Related recommendations:

mysql video tutorial: https://www.php.cn/course/list/51.html

The above is the detailed content of Methods and principles of Redis persistence snapshots. For more information, please follow other related articles on the PHP Chinese website!