What are the situations of Redis blocking?

Command blocking

Using inappropriate commands causes client blocking:

• keys *: Get all key operations;

• Hgetall: Returns the sum of all fields in the hash table;

• smembers: Returns all members in the set;

These commands The time complexity is O(n), and sometimes the entire table is scanned. As n increases, the time complexity will increase and the client will be blocked.

SAVE blocking

Everyone knows that when Redis takes an RDB snapshot, it will call the system function fork() and create a child thread to complete the writing of temporary files, and the triggering condition is save configuration in the configuration file.

When our configuration is reached, the bgsave command will be triggered to create a snapshot. This method will not block the main thread, while manually executing the save command will be executed in the main thread, BlockingMain thread.

Synchronous persistence

When Redis records AOF logs directly, if there are a large number of write operations and is configured for synchronous persistence

appendfsync always

That is, every time a data change occurs, It is recorded to disk immediately. Because writing to disk is time-consuming and has poor performance, it sometimes blocks the main thread.

AOF rewriting

• fork creates a sub-thread to rewrite the file. When executing the BGREWRITEAOF command, the Redis server will maintain an AOF Rewrite buffer, which records all write commands executed by the server during the creation of a new AOF file by the child thread.

• When the child thread completes the work of creating a new AOF file, the server will append all the contents in the rewrite buffer to the end of the new AOF file, so that the new AOF file saves The database state is consistent with the existing database state.

• Finally, the server replaces the old AOF file with the new AOF file to complete the AOF file rewriting operation.

Blocking occurs during step 2. When writing new data in the buffer to a new file, blocking will occur.

AOF log

The logging of AOF does not record the log before executing the command like the relational database (to facilitate fault recovery), but uses the method of executing the command first and then recording the log.

The reason is that AOF logging will not perform syntax checking on commands, which can reduce additional checking overhead and will not block the execution of the current command, but may cause blocking to the next operation. risk.

This is because the AOF log is also executed in the main thread. If the disk writing pressure is high when writing the log file to the disk, the disk writing will be very slow, which will lead to Subsequent operations cannot be performed.

Big Key Problem

Big key does not mean that the value of the key is very large, but that the value corresponding to the key is very large.

The blocking problems caused by large keys are as follows:

• Client timeout blocking: Since Redis execution commands are single-threaded, it will be more time-consuming to operate large keys. , then Redis will be blocked. From the perspective of the client, there will be no response for a long, long time.

• Causes network congestion: each time you obtain a large key, the network traffic generated is large. If the size of a key is 1 MB and the number of visits per second is 1000, then 1000MB will be generated per second. traffic, which is catastrophic for servers with ordinary Gigabit network cards.

• Blocking the worker thread: If you use del to delete a large key, the worker thread will be blocked, making it impossible to process subsequent commands.

Find big key

When we use the --bigkeys parameter that comes with Redis to find the big key, it is best to choose the slave node Execute this command on the master node, because when executed on the master node, it will block the master node.

• We can also use the SCAN command to find big keys;

• In order to identify big keys, you need to first ensure that Redis uses RDB persistence. And analyze the corresponding RDB file. There are ready-made tools on the Internet:

• redis-rdb-tools: A tool written in Python language to analyze the RDB snapshot file of Redis

• This tool is called rdb_bigkeys and is written in Go language. It can be used to analyze Redis' RDB snapshot files and has higher performance.

Delete large key

The essence of the deletion operation is to release the memory space occupied by the key-value pair.

Releasing memory is only the first step. In order to manage memory space more efficiently, when the application releases memory,

the operating system needs to insert the released memory block into a linked list of free memory blocks , for subsequent management and redistribution. This process itself takes a certain amount of time and will blockthe application currently releasing memory.

So, if a large amount of memory is released at once, the operation time of the free memory block linked list will increase, which will accordingly cause the blocking of the Redis

main thread. If the main thread is blocked, All other requests may timeout, and more and more timeouts will cause Redis connections to be exhausted and various exceptions to occur.

When deleting large keys, it is recommended to use batch deletion and asynchronous deletion.

Clear the database

Clearing the database is the same as deleting bigkey above. Flushdb and flushall also involve deleting and releasing all key-value pairs, which are also the blocking points of Redis.

Cluster expansion

Redis cluster can dynamically expand and shrink nodes. This process is currently in a semi-automatic state and requires manual intervention.

When expanding or shrinking, data migration is required. In order to ensure the consistency of migration, Redis all migration operations are synchronous operations.

When performing migration, Redis at both ends will enter a blocking state of varying lengths. For small keys, this time can be ignored, but if the memory usage of the key is too large, serious Sometimes, a failover within the cluster will be triggered, causing unnecessary switching.

The above is the detailed content of What are the situations of Redis blocking?. For more information, please follow other related articles on the PHP Chinese website!