How does springboot integrate redis to modify partitions

springboot integrates redis to modify partitions

The origin of the problem

Recently, springboot is used to integrate redis. A system dynamic data source connects to different databases and caches the redis used. Then the data of different databases needs to be Cache to different partitions of redis, that is, different libraries.

Old version solution

The old version here refers to the one before 2.0. The 1.5.9 I used is ok.

The redis configuration class will not be posted here, there are many online.

1. Use JedisConnectionFactory to modify

@Autowired
JedisConnectionFactory jedisConnectionFactory;
jedisConnectionFactory.setDatabase(database);

2. Use redisTemplate to modify

redisTemplate.getConnectionFactory().getConnection().select(database);

The above two methods do not require redis Bean is specially added to the configuration class

New version solution

The new version here refers to the version after 2.0, I am using 2.0.3

The following needs to be added to the redis configuration class bean

@Bean
    RedisStandaloneConfiguration redisStandaloneConfiguration() {
        RedisStandaloneConfiguration redisStandaloneConfiguration = new RedisStandaloneConfiguration();
        redisStandaloneConfiguration.setHostName("localhost");
        redisStandaloneConfiguration.setPort(6379);
        redisStandaloneConfiguration.setDatabase(0);
        return redisStandaloneConfiguration;
    }
    @Bean
    JedisConnectionFactory jedisConnectionFactory(RedisStandaloneConfiguration redisStandaloneConfiguration) {
        //redisStandaloneConfiguration.setPassword(RedisPassword.of(password));
        JedisClientConfiguration.JedisClientConfigurationBuilder jedisClientConfiguration = JedisClientConfiguration.builder();
        jedisClientConfiguration.connectTimeout(Duration.ofMillis(0));//  connection timeout
        JedisConnectionFactory factory = new JedisConnectionFactory(redisStandaloneConfiguration,
                jedisClientConfiguration.build());
        return factory;
    }

Use RedisStandaloneConfiguration to modify

@Autowired
RedisStandaloneConfiguration redisStandaloneConfiguration;
redisStandaloneConfiguration.setDatabase(database);

redis partition

How data is distributed on multiple Redis instances

Partitioning is to distribute your data On multiple Redis instances so that each instance only contains a portion of the data.

Why partitioning is useful

Redis partitioning has two main goals:

1. It allows larger databases to use the memory of many computers sum. Without partitioning, you are limited to the memory of a single computer.

2. It allows extending computing power to multiple cores and multiple computers, and extending network bandwidth to multiple computers and network adapters.

Suppose we have 4 Redis instances (R0, R1, R2, R3), with many keys representing users, such as user:1, user:2, ... Wait, then we have many ways to store a key.

The simplest way is to partition by range, that is, allocate data to the specified Redis instance according to the mapping range of the object. For example, we stipulate that IDs from 0 to 10000 will be assigned to R0, 10001 to 20000 to R1, and so on. This method is possible, but one disadvantage is that a table is needed to maintain this mapping relationship. This table needs to be managed, and each key requires one such table, so range partitioning in Redis is generally frowned upon because it is much less efficient than other partitioning methods.

In addition to range partitioning, another method is hash partitioning:

Step 1, take the key and apply a hash function to convert it into a number. For example, if key is foobar and the hash function is crc32, then crc32(foobar) will output 93024922.

Step 2. Use modulo operation (modulo) on this number to convert it into a direct number from 0 to 3, so that this number can be mapped to one of my four Redis instances. For example, 93024922 % 4 = 2, so foobar should be stored on the R2 instance.

(PS: First, do a hash operation on the key to get a number, and then take the modulo of this number to determine which instance the final data should be stored on)

The method of partitioning is also There are many kinds, and through the above two examples, you should be able to understand. An advanced form of hash partitioning is called consistent hashing and is implemented by several Redis clients and brokers.

Different partition implementation

Client partition: For a given key, the client directly selects the correct node for reading and writing. Many Redis clients implement client-side partitioning.

Agent partition: The client sends a request to a proxy, and the proxy communicates with Redis. The proxy will select the correct Redis instance based on our configuration.

Query routing: You can send your query to any Redis instance, and the instance will redirect your query to the correct server.

(PS: For a given key, the job of partitioning is to select a correct Redis instance, then this selection process can be done by the client, agent or Redis instance)

Disadvantages of partitioning

1. Operations involving multiple keys are usually not supported. For keys mapped to two different Redis instances, you cannot perform insert operations on them.

2. Redis transactions cannot be used for operations involving multiple keys.

3. The partition granularity is key, so it is impossible to convert a single very huge key (for example, a very large sorted set ) to split data

4. When using partitions, data processing will be more complicated. For instances, you must process multiple RDB/AOF files. In order to back up data, you need to aggregate persistence from multiple instances and hosts. document.

5. Adding and deleting capacity (space) becomes more complicated. For example, Redis Cluster supports transparent data rebalancing that adds and removes nodes at runtime, but other systems such as client partitions and brokers do not support this feature. However, a technique called pre-sharding helps in this regard.

Data storage or cache?

When Redis is used as a data store, a given key must always be mapped to the same Redis instance. When used as a cache it is not a big problem if a given node becomes unavailable.

If the preferred node for a given key is unavailable, consistent hashing implementations are usually able to switch to other nodes. Similarly, if a new node is added, some of the new keys will start to be stored on the new node.

• If you use Redis as a cache, it is easy to scale using consistent hashing.

• If Redis is used as storage, a fixed keys-to-nodes mapping is used, so the number of nodes must be fixed and cannot be changed. Redis Cluster is a viable system if keys need to be rebalanced between nodes.

The above is the detailed content of How does springboot integrate redis to modify partitions. For more information, please follow other related articles on the PHP Chinese website!