Monitoring and optimization of Redis persistence process

Redis persistence process has always been a common factor affecting redis performance. How to monitor persistence and how to optimize the persistence process? Here we take a look.

Monitoring and optimization of fork

No matter which persistence is used, RDB persistence or AOF rewriting, the main process will fork out. A child process, in which the generation of RDB files or the rewriting of AOF are completed. The fork operation is a relatively heavy operation for the operating system. During the fork phase, redis will block for a period of time. The blocking time is directly proportional to the memory size occupied by redis data. Each 1G memory fork takes 20 milliseconds.

If you want to know the blocking time of the fork stage, you can use the info stats command to view the value of the latest_fork_usec option, the unit is microseconds. Remember it's microseconds, not milliseconds.

# redis-cli info stats | grep latest
latest_fork_usec:323

Methods to optimize fork:

• Control the memory size occupied by redis. If the memory usage is too large, the application can be split and deployed on multiple servers to share the memory usage of redis.

• Appropriately reduce the frequency of fork operations.

Memory monitoring

RDB persistence log is as follows:

……
21692:C 15 May 2020 14:17:06.935 * DB saved on disk
21692:C 15 May 2020 14:17:06.936 * RDB: 2 MB of memory used by copy-on-write
……

You can see The RDB persistence process consumes 2M memory.

The AOF persistence log is as follows:

……
15786:C 23 May 2020 07:39:59.145 * AOF rewrite: 2MB of memory used by copy-on-write
10679:M 23 May 2020 07:39:59.201 * Background AOF rewrite terminated with success
10679:M 23 May 2020 07:39:59.201 * Residual parent diff successfully flushed to the rewritten AOF (0.02 MB)
10679:M 23 May 2020 07:39:59.201 * Background AOF rewrite finished successfully

You can see that the memory occupied by aof rewriting is 2MB 0.02MB=2.02MB

If you want to monitor the memory during the persistence process For the occupancy status, you can write a shell script to count the relevant information in the redis log.

Hard disk monitoring

The Redis persistence process will put pressure on the hard disk, because after persistence, the memory data will be saved to the hard disk. .

The Linux system has sar, iostat, etc. commands for monitoring the hard disk. If it is found that the hard disk IO pressure exceeds the threshold, then compare the persistence time according to the redis log to see if it is caused by the pressure of redis persistence. .

Optimization method Here are two points:

• Use a disk with good performance. Mechanical hard drives are definitely not as good as solid-state drives.

• If several redis instances are configured on a single machine, they can be written to different disks to reduce the writing pressure on the disk.

Single-machine multi-instance deployment

Because redis is a single-threaded architecture, if only one redis instance is deployed on a server , then it is a waste for multi-core CPUs. Therefore, multiple redis applications are usually deployed on one server. For example, three redis services are opened, and the port numbers are 6379, 6380, and 6381. 6379 is used for caching services, 6380 is used for message queues, and 6381 is used for tags. and recommendation systems.

This can indeed make full use of the CPU, but it can easily cause problems. If multiple instances are persisting at the same time, the pressure on the CPU, memory and video will be very large. A good practice is to isolate them so that only one instance is persisting at a time.

The pseudo code to achieve this effect is as follows:

while (true)
{
     $redisObj = [6379,6380,……];
     
     foreach ($redisObj as $obj) {
          // 该实例是否构成重写的要求
          if (rewriteConf($ojb)) {
           // 该实例进行持久化
          }
     }
}

foreach is used to traverse each redis instance, and then determine whether the instance meets the conditions for rewriting. If it is met, rewriting will begin. In this way, multiple redis instances can be persisted and isolated.

The above is the detailed content of Monitoring and optimization of Redis persistence process. For more information, please follow other related articles on the PHP Chinese website!