This article will take you to understand the complete version of Redis persistence.

❝

This article explains the knowledge points Introduction to persistence RDB AOF The difference between RDB and AOF Persistence application scenarios

❞

Preface

❝

Kaka compiled a roadmap to create an interview guide, and prepared to write articles according to this roadmap. Later, I found that I was adding knowledge points that were not supplemented. I also look forward to your partners joining in to help add some information. See you in the comments section!

❞

Insert image description here

Demo environment

centos7.0 redis4.0 redis storage directory:/usr/local/redis redis.conf storage directory:/usr/local/redis/data

1. Introduction to persistence

All data in redis is stored in memory. If redis crashes, the data will be lost. Redis persistence is to save data on disk. The working mechanism that uses permanent storage media to save data processes and restore the saved data at a specific time is called persistence.

What is saved in the persistence process?

The first snapshot form stores data results and focuses on the data, which is the RDB discussed below

The second operation process stores the operation process. The storage structure is complex and the focus is The point is the data operation process, which is the AOF

#2 discussed below. RDB

##2-1 RDB startup mode -- save command

The following figure is the configuration information of redis.conf. After executing save, a The file of dump.rdb

Now we set a value and then save it. There will be a file of dump6379.rdb under /usr/local/redis/data

2-2 RDB startup mode -- save command related configuration

• dbfilename dump6379.rdb: Set the local database file name, the default value is dump.rdb
• dir: The path to store the rdb file
• rdbcompression yes: Set the storage to local Whether to compress data in the database, the default is yes, using lzf compression
• rdbchecksum yes: Set whether to process RDB file format verification, the verification process is performed during both writing and reading files.

2-3 RDB data recovery

In fact, this data recovery is different from other relationships There is basically no need to do anything to restore a large database. Just restart it

2-4 RDB -- How the save command works

This picture Sourced from online videos. The execution of the save command will block the current redis server until the current RDB process is completed, which may cause long-term blocking. This command is basically abandoned and no longer used during the work process. Will replace all

with bgsave

2-5 RDB -- Working principle of bgsave instruction

When bgsave is executed in redis, it will return directly A Background saving started

At this time we are taking a look at the log file. The bgsave command is optimized for the save blocking problem

##2-5 RDB -- Configuration file auto-start

<span style="display: block; background: url(https://my-wechat.mdnice.com/point.png); height: 30px; width: 100%; background-size: 40px; background-repeat: no-repeat; background-color: #272822; margin-bottom: -7px; border-radius: 5px; background-position: 10px 10px;"></span><code class="hljs" style="overflow-x: auto; padding: 16px; color: #ddd; display: -webkit-box; font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; font-size: 12px; -webkit-overflow-scrolling: touch; letter-spacing: 0px; padding-top: 15px; background: #272822; border-radius: 5px;"><span class="hljs-selector-tag" style="color: #f92672; font-weight: bold; line-height: 26px;">save</span> 900 1<br/><span class="hljs-selector-tag" style="color: #f92672; font-weight: bold; line-height: 26px;">save</span> 300 10<br/><span class="hljs-selector-tag" style="color: #f92672; font-weight: bold; line-height: 26px;">save</span> 60 10000<br/><span class="hljs-selector-tag" style="color: #f92672; font-weight: bold; line-height: 26px;">stop-writes-on-bgsave-error</span> <span class="hljs-selector-tag" style="color: #f92672; font-weight: bold; line-height: 26px;">yes</span><br/></code>

save [Time] [Number of key changes]

That is to say, there are 10 in 300 seconds If the key value changes, bgsave

will be executed in the background.

3. AOF

##3-1 AOF concept

AOF persistence: Record each write command in an independent log, and re-execute the commands in the AOF file during restart to achieve data recovery. Compared with RDB, it can be simply described as the process of recording data generation

The main function of AOF is to solve the real-time nature of data persistence, and it is currently the mainstream method of redis persistence

3-2 AOF data writing process

Execute a redis command

redis’s AOF will refresh the command buffer Area

Then synchronize to the .aof file configured in redis.conf according to certain policies

3-3 Three strategies for AOF writing data

• always: every write operation are synchronized to the AOF file, with zero data error and low performance. It is not recommended to use
• everysec: Synchronize the instructions in the buffer to the AOF file every second, and the data accuracy is relatively low. High, with higher performance, is recommended and is also the default configuration.However, if the system suddenly crashes, the data within 1 second will be lost.
• no: The operating system controls the cycle of each synchronization to the AOF file, and the overall process is uncontrollable

3-4 AOF function enabled

• Configuration: appendonly yes|no
• Function: Whether to enable AOF persistence function, the default is not enabled
• Configuration: appendfsync always| everysec | no
• Function: AOF write data strategy
• Configuration: appenfilename filename
• Function: AOF persistence file name, the default name is appendonly.aof

Then use restart the redis service, you can use it in usr/local/redis/data You can see the appendonly.aof file in the directoryThen we execute a command on the redis client and check it out. You can see that the data will be stored in the appendonly.aof file.

3-5 Problems with AOF writing data

Let’s look at a case first. After we repeatedly set the name key , open the appendonly.aof file to view, you can see that there are three operations, but these three operations are all modified by one key! Can't we only save the last key? With this question, we continue to look down

3-6 AOF rewriting

As commands continue to be written to AOF, the file will become larger and larger. In order to solve this problem, redis introduces the AOF rewriting mechanism to compress the file size. AOF file rewriting is the process of converting data in the redis process into write commands and synchronizing them to the new AOF file. Simply put, it converts the execution results of several commands on the same data into the execution records of the instructions corresponding to the final result data.

For example, we executed the set name command three times above, but in the end we only need the data of the last execution. That is, we only need the last execution record.

3-7 AOF rewriting function

• Reduce disk usage and improve disk utilization
• Improve persistence efficiency, reduce persistence write time, and improve IO performance
• Reduce data recovery time and improve data recovery efficiency

3-8 AOF rewrite rules

• The process has timed out Data is no longer written to the file
• Ignore invalid instructions and use in-process data to generate directly during rewriting, so that the new AOF file value retains the final data writing command. Such as del instruction, hdel, srem. Set a key value multiple times, etc.
• Multiple write commands for the same data are merged into one command: such as lpush list a lpush lsit b lpush list c can be converted For lpush list a b c.However, in order to prevent client buffer overflow caused by excessive data volume, each instruction of list, set, hash, zset types can write up to 64 elements

3-9 AOF manual rewriting

Command: bgrewriteaof

Then we 3- For the question 5, we execute the bgrewriteaof command on the command line and then check the appendonly.aof file

After the execution, we will find that the file has become smaller and there is only one command in the file

Insert picture description here

3-10 AOF manual rewriting working principle

Insert image description here

3-11 AOF automatic rewrite

Configuration: auto-aof-rewrite-percentage 100 | auto-aof-rewrite-min-size 64mbTrigger comparison parameters: aof_current_size | aof_base_size

When aof_current_size > auto-aof-rewrite-min-size 64mb will start rewriting

This picture comes from the Internet

3-11 AOF workflow and rewrite flow = process

##4. The difference between RDB and AOF

• ## is very sensitive to data, it is recommended to use the default AOF Persistence solution

  AOF persistence strategy uses everysecond, fsync-times per second • This strategy redis can still maintain good processing performance. When a problem occurs, up to 0- Data within 1 second.
  Note: Due to the large storage size of AO files and the slow recovery speed
• Validity of data presentation stage, it is recommended to use RDB persistence solution

  The data can be well maintained without loss during the stage (this stage is for developers to operate and maintain Manually maintained), and the recovery speed is faster. The RDB solution is usually used for stage point data recovery
  Note: Using RDB to achieve tight data persistence will cause Redis to drop very low
• Comprehensive comparison
  • The choice between RDB and AOF is actually a trade-off, each has advantages and disadvantages
  • If you cannot bear data loss within a few minutes , very sensitive to industry data, choose A0F
  • . If you can withstand data loss within a few minutes, if you pursue the recovery speed of large data sets, choose RDB
  • Use RDB for disaster recovery
  • Double insurance strategy, start RDB and AOF at the same time. After restarting, Redis gives priority to using A0F to recover data, reducing the amount of lost data

  ##❝Persistence in learning, persistence in blogging, and persistence in sharing are the beliefs that Kaka has always adhered to since his career. I hope that Kaka’s articles can be seen in the huge Internet Bringing you a little help. See you in the next issue.

  ❞

## Recommended: "

redis tutorial

》

The above is the detailed content of This article will take you to understand the complete version of Redis persistence.. For more information, please follow other related articles on the PHP Chinese website!