Partitioning is the process of splitting data into multiple Redis instances, so each instance only saves a subset of keys. This article will introduce to you how redis implements partitioning.
Why do we need to partition? What was the motivation for partition? Generally speaking, the benefits of Redis partitioning are roughly as follows:
1. Performance improvement. The network I/O capabilities and computing resources of single-machine Redis are limited, and requests are distributed to multiple machines. Making full use of the computing power and network bandwidth of multiple machines helps improve the overall service capabilities of Redis.
2. Horizontal expansion of storage. Even if the service capabilities of Redis can meet application needs, as the storage data increases, a single machine is limited by the storage capacity of the machine itself, and the data is dispersed to multiple machines. Upper storage enables the Redis service to scale horizontally.
In general, partitioning makes our original problem of being limited by the hardware resources of a single computer no longer a problem. Not enough storage? Not enough computing resources? Not enough bandwidth? We can all solve these problems by adding more machines.
Redis Partition Basics
There are many specific strategies for partitioning in actual applications. For example, suppose we already have a set of four Redis instances, namely R0, R1, R2, R3. In addition, we have a batch of keys representing users, such as: user:1, user:2,...and so on. The number after "user:" represents the user's ID. What we need to do These keys are stored in four different Redis instances.
How to do it? The simplest way is range partitioning. Let's take a look at how to do it based on range partitioning.
Range partitioning
The so-called range partitioning is to map all keys in a range to the same Redis instance. Adding the data set is still the user data mentioned above. , the specific method is as follows:
We can map user data with user IDs from 0 to 10000 to R0 instances, and map objects with user IDs from 10001 to 20000 to R1 instances, and so on.
Although this method is simple, it is very effective in practical applications, but there are still problems:
1. We need a table, which is used to store Mapping relationship between user ID range and Redis instance. For example, user ID 0-10000 is mapped to R0 instance...
2. We not only need to maintain this table, but we also need such a table for each object type. For example, we are currently storing user information. If we are storing order information, we will Another mapping table needs to be created.
3. What if the key of the data we want to store cannot be divided according to the range. For example, our key is a set of uuid. At this time, it is difficult to use range partitioning.
Hash partition
An obvious advantage of hash partition compared to range partition is that hash partition is suitable for any form of key, unlike range partitioning. The form of key is object_name:
id=hash(key)%N
where id represents the number of the Redis instance. The formula describes the first step based on key and a hash function. (such as crc32 function) calculates a numeric value. Following the above example, the first key we want to process is user:1, and the result of hash (user:1) is 93024922.
Then the hash result is modulo. The purpose of modulo is to calculate a value between 0 and 3, so this value can be mapped to one of our Redis instances. For example, if the result of 93024922%4 is 2, we will know that foobar will be stored on R2.
Different partition implementations
Partitions can be implemented in different parts of the redis software stack. Let’s take a look at the following:
Client implementation
Client implementation means that the key is determined on the redis client in which Redis instance it will be stored in, as shown in the figure below:
Proxy implementation
Proxy implementation means that the client sends the request to the proxy server. The proxy server implements the Redis protocol, so the proxy server can proxy the communication between the client and the Redis server. The proxy server forwards the client's request to the correct Redis instance through the configured partition schema, and returns the feedback message to the client.
The schematic diagram of the agent's implementation of Redis partition is as follows:
Query routing
Query routing is an implementation of Redis Cluster A Redis partitioning method:
During the query routing process, we can randomly send the query request to any Redis instance. This Redis instance is responsible for forwarding the request to the correct In the Redis instance. Redis cluster implements a hybrid that cooperates with the client for query routing.
Disadvantages of Redis partition
Although Redis partitioning is so far so good so far, Redis partitioning has some fatal shortcomings, which causes some Redis functions to not work well in a partitioned environment. Let’s take a look:
1. Multi-key operations are not supported. For example, the keys we want to operate in batches are mapped to different Redis instances.
2. Multi-key Redis transactions are not supported.
3. The minimum granularity of partitioning is the key, so we cannot map a large data set associated with a key to different instances.
4. When applying partitioning, data processing is very complex. For example, we need to process multiple rdb/aof files and gather files distributed in different instances for backup.
5. Adding and deleting machines is very complex. For example, Redis cluster supports almost runtime transparent rebalancing required to add or reduce machines. However, methods such as client and proxy partitioning are not supported. of this function.
Persistent storage or caching
Although data partitioning is conceptually the same for Redis, whether it is data persistent storage or caching, however, for data Persistent storage still has a big limitation.
When we use Redis as persistent storage, each key must always be mapped to the same Redis instance. When Redis is used as a cache, for this key, if one instance cannot be used, this key can also be mapped to other instances.
Consistent hashing implementations usually make it possible to map a key to another instance when the instance to which the key is mapped becomes unavailable. Similarly, if a machine is added, part of the keys will be mapped to the new machine. Two points we need to understand are as follows:
1. If Redis is used as a cache, and the requirements are easy Adding or removing machines is very simple using consistent hashing.
2. If Redis is used as (persistent) storage, a fixed key-to-instance mapping is required, so we can no longer flexibly add or delete machines. Otherwise, we need the system to be able to rebalance when adding or deleting machines, which is currently supported by Redis Cluster.
Pre-Sharding
Through the above introduction, we know that there are problems with the application of Redis partition. Unless we only use Redis as a cache, it will be difficult to add machines or Deleting a machine is very troublesome.
However, usually our Redis capacity changes are very common in practical applications. For example, I need 10 Redis machines today, and I may need 50 machines tomorrow.
Given that Redis is a very lightweight service (each instance only occupies 1M), a simple solution to the above problem is:
We can open multiple Even though the Redis instance is a physical machine, we can start multiple instances at the beginning. We can choose some instances, such as 32 or 64 instances, as our working cluster. When one physical machine does not have enough storage, we can move the general instances to our second physical machine and pair them in sequence. We can ensure that the number of Redis instances in the cluster remains unchanged and achieve the purpose of expanding the machine.
How to move a Redis instance? When we need to move the Redis instance to an independent machine, we can do it through the following steps:
1. Start a new Redis instance on the new physical machine.
2. Use the new physical machine as the slave machine to be moved.
3. Stop the client.
4. Update the IP address of the Redis instance to be moved.
5. Send the SLAVEOF ON ONE command to the slave machine.
6. Use the new IP to start the Redis client.
7. Close the Redis instance that is no longer in use.
For more redis knowledge, please pay attention to the redis introductory tutorial column.
The above is the detailed content of Redis partition implementation. For more information, please follow other related articles on the PHP Chinese website!

Redis是现在最热门的key-value数据库,Redis的最大特点是key-value存储所带来的简单和高性能;相较于MongoDB和Redis,晚一年发布的ES可能知名度要低一些,ES的特点是搜索,ES是围绕搜索设计的。

本篇文章给大家带来了关于redis的相关知识,其中主要介绍了关于redis的一些优势和特点,Redis 是一个开源的使用ANSI C语言编写、遵守 BSD 协议、支持网络、可基于内存、分布式存储数据库,下面一起来看一下,希望对大家有帮助。

本篇文章给大家带来了关于redis的相关知识,其中主要介绍了Redis Cluster集群收缩主从节点的相关问题,包括了Cluster集群收缩概念、将6390主节点从集群中收缩、验证数据迁移过程是否导致数据异常等,希望对大家有帮助。

本篇文章给大家带来了关于redis的相关知识,其中主要介绍了Redis实现排行榜及相同积分按时间排序,本文通过实例代码给大家介绍的非常详细,对大家的学习或工作具有一定的参考借鉴价值,希望对大家有帮助。

本篇文章给大家带来了关于redis的相关知识,其中主要介绍了关于原子操作中命令原子性的相关问题,包括了处理并发的方案、编程模型、多IO线程以及单命令的相关内容,下面一起看一下,希望对大家有帮助。

本篇文章给大家带来了关于redis的相关知识,其中主要介绍了bitmap问题,Redis 为我们提供了位图这一数据结构,位图数据结构其实并不是一个全新的玩意,我们可以简单的认为就是个数组,只是里面的内容只能为0或1而已,希望对大家有帮助。

本篇文章给大家带来了关于redis的相关知识,其中主要介绍了Redis实现排行榜及相同积分按时间排序,本文通过实例代码给大家介绍的非常详细,下面一起来看一下,希望对大家有帮助。

本篇文章给大家带来了关于redis的相关知识,其中主要介绍了关于实现秒杀的相关内容,包括了秒杀逻辑、存在的链接超时、超卖和库存遗留的问题,下面一起来看一下,希望对大家有帮助。


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

AI Hentai Generator
Generate AI Hentai for free.

Hot Article

Hot Tools

Zend Studio 13.0.1
Powerful PHP integrated development environment

Atom editor mac version download
The most popular open source editor

ZendStudio 13.5.1 Mac
Powerful PHP integrated development environment

SublimeText3 Mac version
God-level code editing software (SublimeText3)

Dreamweaver Mac version
Visual web development tools
