How to solve Redis cache avalanche, breakdown and penetration

This article brings you relevant knowledge about Redis, which mainly introduces how to solve the problems related to redis cache avalanche, breakdown and penetration. Cache avalanche refers to a large number of requests. The cached data in Redis cannot be hit, that is, the data cannot be found in Redis. Let's take a look at it, I hope it will be helpful to everyone.

Recommended learning: Redis video tutorial

1. Cache avalanche

1. What is Cache avalanche?

Cache avalanche refers to a large number of requests that cannot hit the cached data in Redis, that is, the data cannot be found in Redis , then the business system can only query the database, which will cause all requests to be sent to the database. As shown in the figure below:

The database is not as capable of handling a large number of requests as Redis. The surge in requests caused by cache avalanche will definitely cause the database to go down. , This will inevitably affect the business system, so if a cache avalanche occurs, it will definitely be fatal to the business system.

2. Why does cache avalanche occur?

Under what circumstances does a cache avalanche occur? To sum up, there are two reasons:

• A large number of Redis cached data expired at the same time, causing all requests sent to Redis to fail. Hit data can only be queried in the database.

• RedisThe server is down, all requests cannot be processed through Redis, and can only be turned to the database to query data.

3. How to avoid cache avalanche?

There are different solutions to the causes of cache avalanche:

• For a large number of random cache expiration times, the solution is to add the original expiration time to Add a random expiration time, such as a random expiration time between 1 and 5 minutes, to avoid a large amount of cached data expiring at the same time.

• To solve the cache avalanche caused by Redis, you can set up the Redis master-slave server in advance for data synchronization, and Configure the sentinel mechanism so that when the Redis server is unable to provide services due to downtime, the sentinel can set the Redis slave server to the master server and continue to provide services.

2. Cache breakdown

1. What is cache breakdown

Cache breakdown is similar to cache avalanche. The avalanche is caused by a large number of The data has expired, and cache breakdown refers to the expiration of hotspot data. All requests for hotspot data need to be processed in the database, as shown in the following figure:

2 . How to avoid cache penetration?

Three ways to solve cache breakdown:

• Do not set expiration time

If we can know in advance that a certain data is hot data, then You can not set the expiration of these data to avoid cache breakdown problems. For example, some products in flash sales will be accessed by a large number of users during the flash sale. At this time, we can write the product data for flash sales into the cache in advance and not Set expiration time.

• Mutex lock

If we know in advance that certain data will be accessed in large quantities, of course we can set it to not expire, but more often than not, we cannot predict it in advance. How to deal with this situation?

Let’s analyze the cache breakdown situation:

Under normal circumstances, when a certain Redis cached data expires, if there is a request for the data, then Re-query in the database and write to the cache so that subsequent requests can hit the cache without querying the database again.

When the hotspot data expires, due to a large number of requests, when a request cannot hit the cache, the database will be queried and the data will be rewritten to Redis, that is, writing RedisBefore, when other requests came in, the database would also be queried.

Okay, we know that after the hotspot data expires, many requests will query the database, so we can add a mutex lock to the business logic of querying the database. Only requests that obtain the lock can query the database and put The data is written back to Redis, while other requests that have not obtained the lock can only wait for the data to be ready.

The above steps are shown in the figure below:

• Set logical expiration time

Although using a mutex lock can solve the cache breakdown problem very simply, requests that do not obtain the lock are queued up, which affects the performance of the system. , and another way to solve cache breakdown is to add an expiration time to the business data redundancy. For example, in the following data, we have added the expire_at field to indicate the data expiration time.

{"name":"test","expire_at":"1599999999"}复制代码

The implementation process of this method is shown in the figure below:

The hotspot data in the cache has a redundant logical expiration time, but the data is RedisDo not set expiration time

When a request gets the data in Redis, determine whether the logical expiration time has expired. If it has not expired, return directly. If When it expires, another thread is opened to obtain the lock, query the database and write the latest queried data back to Redis, and the current request returns the queried data.

3. Cache penetration

1. What is cache penetration

Cache penetration means that the data to be found is neither in the cache nor in the database, because It is not in the cache, so the request will definitely reach the database. RedisThe cache is in name only, as shown in the following figure:

2. Why does cache penetration occur

Under what conditions will cache penetration occur? There are mainly three situations:

• Malicious user attack request

• Malicious operation of Redis and the data in the database Deleted

• When the user has not yet generated content, such as the user's article list, the user has not written an article, so there is no data in the cache and database

3. How to avoid cache penetration?

a. Cache empty value or default value

When no data can be queried in the Redis cache, query it from the database again. If there is no data, directly Cache a space or default value to avoid querying the database next time; however, in order to prevent the database from responding to the database and then returning a null value, the expiration time should be set for the cache, or the corresponding data should be cleared directly when the data is generated. Cache empty values.

b. Bloom filter

Although caching null values ​​can solve the cache penetration problem, it still needs to query the database once to determine whether there is data. If there is a malicious attack by a user, high concurrency Querying using data IDs that do not exist in the system requires all queries to go through the database, which will still put a lot of pressure on the database.

So, is there any way to determine whether the data exists without querying the database? Yes, use Bloom filter.

The Bloom filter mainly consists of two parts: bit array N hash functions. The principle is:

• Use N hash functions to mark the The data is hashed.

• Take the calculated hash value modulo the length of the bit array, so that the position of each hash value in the bit array can be obtained.

• Mark the corresponding position in the bit array as 1.

The following is a schematic diagram of the Bloom filter principle:

When data is to be written, perform the steps described above and calculate Corresponds to the bit array position and is marked as 1, then when executing the query, you can check whether the data exists.

In addition, due to errors caused by hash collision problems, non-existent data will be judged to exist after passing through the Bloom filter, and then the database will be checked. However, the probability of hash collision is very small. Using Bloom filters can already help us intercept most penetration requests.

Redis itself supports Bloom filters, so we can directly use Redis Bloom filters without having to implement them ourselves, which is very convenient.

4. Summary

Cache avalanche, breakdown, and penetration are cache exception problems that are often encountered when caching business applications. The causes and solutions are as follows:

Problem	Cause	Solution
Cache Avalanche	A large amount of data expires orRedisserver crashes	1. Random expiration time 2. Master-slave sentinel cluster
Cache breakdown	Hotspot data expiration	1. Do not set expiration time 2. Add mutex lock 3. Redundant logical expiration time
Cache penetration	Request data that is not available in the database and Redis	1. Cache empty value or default value 2. Bloom filter

Recommended learning: Redis video tutorial

The above is the detailed content of How to solve Redis cache avalanche, breakdown and penetration. For more information, please follow other related articles on the PHP Chinese website!