Rumah >pangkalan data >tutorial mysql >Redis Essentials 读书笔记

Redis Essentials 读书笔记

WBOYasal: 2016-06-07 14:50:531393semak imbas

Chapter 9. Redis Cluster and Redis Sentinel (Collective Intelligence) 上一章介绍了复制，一个master可以对应一个或多个slave(replica), 在以下的情况下是够用的： 1. master有足够内存容纳所有key 2. 通过slave可以扩展读，解决网络吞吐量的问题 3. 允

Chapter 9. Redis Cluster and Redis Sentinel (Collective Intelligence)

上一章介绍了复制，一个master可以对应一个或多个slave(replica), 在以下的情况下是够用的：
1. master有足够内存容纳所有key
2. 通过slave可以扩展读，解决网络吞吐量的问题
3. 允许停止master的维护窗口时间
4. 通过slave做数据冗余

但复制解决不了自动failover和自动resharding的问题，在以下的情况不适用：
1. 数据集大于master实例的容量
2. 应用不能停，master没有维护窗口时间
3. 需要分布数据到多个节点
4. 单点故障不可接受

2011年, Salvatore Sanfilippo开始着手解决automatic failover的问题，并建立了项目Redis Sentinel（哨兵，岗哨）。

Redis Sentinel 和 Redis Cluster有共同点, 但分布解决不同的问题。Sentinel的目标是提供可靠的master/slave自动切换，无需sharding数据；而Cluster的目标是在多个实例分布数据，并在master发生故障时自动切换。

Redis Sentinel在2013年发布了Redis 2.8稳定版, 而Redis Cluster在2015中的Redis 3.0提供。

The CAP theorem

分布式系统通常支撑CAP理论，指分布式系统不能同时满足以下条件：
1. Consistency: A read operation is guaranteed to return the most recent write
2. Availability: Any operation is guaranteed to receive a response saying whether it has succeeded or failed
3. Partition tolerance: The system continues to operate when a network partition occurs

由于Redis Sentinel和Redis Cluster都是分布式系统, 它们也遵循CAP理论。网络分区不可避免，因此它们不是CP就是AP。
也就是说，当网络分区发生时，Redis Sentinel 和 Redis Cluster不能同时保证可用性和一致性，但可以有方法使其影响最小化。

They cannot provide availability because there is a quorum that needs to agree on a master election, and depending on the quorum’s decision, part of the system may become unavailable.

They cannot provide consistency under network partitions, for example, when two or more partitions accept writes at the same time. When the network heals and the partitions are joined, some of those writes will be lost (conflicts are not automatically solved, nor are they exposed for clients).

CAP FAQ可参见: http://henryr.github.io/cap-faq/.

Redis Sentinel

Redis Sentinel 是一个分布式系统，在master失效时，可以自动的将slave升级为master（在Sentinel出现之前，这些都是手工做的）。但Sentinel不能分布数据，master必须具有所有的数据，而其它都是副本。

典型的安装是每个Redis服务器上安装一个Sentinel，它是一个独立的进程，在自己的端口监听。Sentinel之间通过订阅频道sentinel:hello相互通讯，同时Sentinel可以监控所有的instance。

The major difference when using Redis Sentinel is that it implements a different interface, which may require you to install a Redis client that supports Sentinel.

client总是连接到Redis实例，但需要查询Sentinel以确定连接到哪个实例。

Redis安装后，有Sentinel配置文件sentinel.conf。在最初的配置中，只列出master，然后Sentinel启动，master会告知Sentinel有哪些slave，这时才会把可用的slave写入配置文件。failover时，配置文件也会重写。

The basic Sentinel configuration

Redis Sentinel配置中总是会通过IP+端口监控master。例如下面的第一行，其中2表示quorum, 也就是最少需要的实例数，以达成一致来选举新的master:

<code class=" hljs lasso">sentinel monitor mymaster <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span> <span class="hljs-number">6379</span> <span class="hljs-number">2</span>
sentinel down<span class="hljs-attribute">-after</span><span class="hljs-attribute">-milliseconds</span> mymaster <span class="hljs-number">30000</span>
sentinel failover<span class="hljs-attribute">-timeout</span> mymaster <span class="hljs-number">180000</span>
sentinel parallel<span class="hljs-attribute">-syncs</span> mymaster <span class="hljs-number">1</span></code>

当master不能访问超过时间down-after-milliseconds，则Sentinel会通知其它Sentinel。

当master变化或新的实例加入（拓扑发生变化），Sentinel配置文件会重写。

failover-timeout 指重新选举前，必须等待的时间，以避免master只是很短期的故障。

Connecting to Sentinel

以下是Ruby Redis client连接Sentinel的配置文件：

<code class=" hljs ruby"><span class="hljs-constant">SENTINELS</span> = [
    {<span class="hljs-symbol">:host</span> => <span class="hljs-string">"127.0.0.1"</span>, <span class="hljs-symbol">:port</span> => <span class="hljs-number">26380</span>},
    {<span class="hljs-symbol">:host</span> => <span class="hljs-string">"127.0.0.1"</span>, <span class="hljs-symbol">:port</span> => <span class="hljs-number">26381</span>}
]
redis = <span class="hljs-constant">Redis</span>.new(<span class="hljs-symbol">:url</span> => <span class="hljs-string">"redis://mymaster"</span>, <span class="hljs-symbol">:sentinels</span> => <span class="hljs-constant">SENTINELS</span>, <span class="hljs-symbol">:role</span> => <span class="hljs-symbol">:master</span>)</code>

Sentinel列表中有2个Sentinel服务，如果第一个失效，client可以连接第二个。role表示连接的目标是master还是任意一个slave。

Network partition (split-brain)

当网络分区发生时，Sentinel不能保证数据一致性，数据可能会丢失。参见Kyle Kingsbury的文章以及Salvatore Sanfilippo的答复

例如，一个master和两个slave，每一个实例都配备一个监控的Sentinel。一个client连接到master写数据。
如果发生网络分区，master一个分区，两个slave形成分区。这时Quorum是2，两个slave中选举出一个新的master。而这时，client继续向之前的master写数据（为什么，Sentinel不是可以重定向吗?）。
如果网络恢复，Sentinel会将之前的master作为选举出的新的master的slave，这时，所有client之前写的数据都丢失了。

Sentinel解决了自动failover的问题，但并没有解决数据在多个Redis实例中分布的问题。而且此failover只是Redis Server的自动切换，客户端呢?
Redis Cluster解决了上面两个问题，下章介绍。

Redis Cluster

Redis Cluster可以在多个实例自动分布数据。并在网络分区时提供可用性，但不提供一致性。

和Sentinel不一样，Redis Cluster只需要运行一个进程，不过需要两个端口，第一个用来服务client，第二个用于节点间的通讯（TCP/IP协议），如错误监控，failover，重新分片等。

节点间通讯的端口=服务client的端口 + 10000

Redis Cluster需要至少3个master，所有的数据在3个master间分片，并复制到slave。建议每一个master都配一个slave，否则数据会丢失

和Sentinel不一样，当failover发生时，只有失效的master上的key不可用，直到它的slave被升级为master。

当Redis运行于cluster模式时，客户端的接口发生了改变，redis-cli需要指定-c选项，否则仍运行在单实例模式，例如：

<code class=" hljs lasso">$ redis<span class="hljs-attribute">-cli</span> <span class="hljs-attribute">-c</span> <span class="hljs-attribute">-h</span> localhost <span class="hljs-attribute">-p</span> <span class="hljs-number">6379</span></code>

Hash slots

分区的方法采用CRC-16 hash方法，所有的master分担16384个固定slot的一部分。

<code class=" hljs vbnet">HASH_SLOT = CRC16(<span class="hljs-keyword">key</span>) <span class="hljs-keyword">mod</span> <span class="hljs-number">16384</span></code>

没有分配slot的master不能存数据，连接到这个master的client会重定向到其它master
所有分配的slot的总和必须为16384
slot的重新分布必须手工做

Hash tags

所有的多key操作要求所有的可以存于同一节点，在Redis Cluster中，只有hash tag才能实现这一点。这和twemproxy类似。hash tag可以保证不同的key最终落到同一slot上。下例中，所有的用户具有相同的hash tag，即{user123}，最终会存到同一slot。

<code class=" hljs ruby"><span class="hljs-constant">SADD</span> {user123}<span class="hljs-symbol">:friends</span><span class="hljs-symbol">:usa</span> <span class="hljs-string">"John"</span> <span class="hljs-string">"Bob"</span>
<span class="hljs-constant">SADD</span> {user123}<span class="hljs-symbol">:friends</span><span class="hljs-symbol">:brazil</span> <span class="hljs-string">"Max"</span> <span class="hljs-string">"Hugo"</span>
<span class="hljs-constant">SUNION</span> {user123}<span class="hljs-symbol">:all_friends</span> {user123}<span class="hljs-symbol">:friends</span><span class="hljs-symbol">:usa</span> {user123}<span class="hljs-symbol">:friends</span><span class="hljs-symbol">:brazil</span></code>

Creating a basic cluster

Redis安装后，在utils/create-cluster目录有一个create-cluster shell脚本，可用来创建简单的cluster，管理cluster可以用redis-trib，包括增删节点，分布slot，监控cluster等。

首先通过create-cluster start初始化6个实例:

<code class=" hljs mel">[redis<span class="hljs-variable">@tt12c</span> create-<span class="hljs-keyword">cluster</span>]$ ./create-<span class="hljs-keyword">cluster</span> start
Starting <span class="hljs-number">30001</span>
Starting <span class="hljs-number">30002</span>
Starting <span class="hljs-number">30003</span>
Starting <span class="hljs-number">30004</span>
Starting <span class="hljs-number">30005</span>
Starting <span class="hljs-number">30006</span>
[redis<span class="hljs-variable">@tt12c</span> create-<span class="hljs-keyword">cluster</span>]$ <span class="hljs-keyword">pwd</span>
/home/redis/redis-<span class="hljs-number">3.0</span><span class="hljs-number">.7</span>/utils/create-<span class="hljs-keyword">cluster</span></code>

然后通过create命令创建cluster。此命令为每一个master配一个slave（注意create命令用到了ruby client，因此必须先安装ruby-client，详见第五章）。
然后将16384的slot均分给3个master。

<code class=" hljs sql">curl -L https://get.rvm.io | bash -s stable <span class="hljs-comment">--ruby <- 先安装ruby最新版, 否则下面的命令会hang</span>
gem install redis

$ ./<span class="hljs-operator"><span class="hljs-keyword">create</span>-cluster <span class="hljs-keyword">create</span>
>>> Creating cluster
>>> Performing hash slots allocation <span class="hljs-keyword">on</span> <span class="hljs-number">6</span> nodes...
<span class="hljs-keyword">Using</span> <span class="hljs-number">3</span> masters:
<span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">30001</span>
<span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">30002</span>
<span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">30003</span>
Adding replica <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">30004</span> <span class="hljs-keyword">to</span> <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">30001</span>
Adding replica <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">30005</span> <span class="hljs-keyword">to</span> <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">30002</span>
Adding replica <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">30006</span> <span class="hljs-keyword">to</span> <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">30003</span>
M: <span class="hljs-number">8785</span>b2942c77261b12bd34cda93ccd4a6ee887e1 <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">30001</span>
   slots:<span class="hljs-number">0</span>-<span class="hljs-number">5460</span> (<span class="hljs-number">5461</span> slots) master
M: <span class="hljs-number">3</span>f79c8fab3a163dcefb45098d405f5806bf6be40 <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">30002</span>
   slots:<span class="hljs-number">5461</span>-<span class="hljs-number">10922</span> (<span class="hljs-number">5462</span> slots) master
M: e0701859f47b7c054b164b4cf5102798192f2500 <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">30003</span>
   slots:<span class="hljs-number">10923</span>-<span class="hljs-number">16383</span> (<span class="hljs-number">5461</span> slots) master
S: <span class="hljs-number">02</span>f0cd798cd3592d5585cbefce471c172525be84 <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">30004</span>
   replicates <span class="hljs-number">8785</span>b2942c77261b12bd34cda93ccd4a6ee887e1
S: <span class="hljs-number">9</span>d1451b5f98244020e47b341019b1576e0421b71 <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">30005</span>
   replicates <span class="hljs-number">3</span>f79c8fab3a163dcefb45098d405f5806bf6be40
S: f5d5b4a411a8197a9921bcdeb35c4db2a4b7c3f9 <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">30006</span>
   replicates e0701859f47b7c054b164b4cf5102798192f2500
Can I <span class="hljs-keyword">set</span> the above configuration? (type <span class="hljs-string">'yes'</span> <span class="hljs-keyword">to</span> accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch <span class="hljs-keyword">to</span> <span class="hljs-keyword">each</span> node
>>> Sending CLUSTER MEET messages <span class="hljs-keyword">to</span> <span class="hljs-keyword">join</span> the cluster
Waiting <span class="hljs-keyword">for</span> the cluster <span class="hljs-keyword">to</span> <span class="hljs-keyword">join</span>..
>>> Performing Cluster <span class="hljs-keyword">Check</span> (<span class="hljs-keyword">using</span> node <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">30001</span>)
M: <span class="hljs-number">8785</span>b2942c77261b12bd34cda93ccd4a6ee887e1 <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">30001</span>
   slots:<span class="hljs-number">0</span>-<span class="hljs-number">5460</span> (<span class="hljs-number">5461</span> slots) master
M: <span class="hljs-number">3</span>f79c8fab3a163dcefb45098d405f5806bf6be40 <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">30002</span>
   slots:<span class="hljs-number">5461</span>-<span class="hljs-number">10922</span> (<span class="hljs-number">5462</span> slots) master
M: e0701859f47b7c054b164b4cf5102798192f2500 <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">30003</span>
   slots:<span class="hljs-number">10923</span>-<span class="hljs-number">16383</span> (<span class="hljs-number">5461</span> slots) master
M: <span class="hljs-number">02</span>f0cd798cd3592d5585cbefce471c172525be84 <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">30004</span>
   slots: (<span class="hljs-number">0</span> slots) master
   replicates <span class="hljs-number">8785</span>b2942c77261b12bd34cda93ccd4a6ee887e1
M: <span class="hljs-number">9</span>d1451b5f98244020e47b341019b1576e0421b71 <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">30005</span>
   slots: (<span class="hljs-number">0</span> slots) master
   replicates <span class="hljs-number">3</span>f79c8fab3a163dcefb45098d405f5806bf6be40
M: f5d5b4a411a8197a9921bcdeb35c4db2a4b7c3f9 <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">30006</span>
   slots: (<span class="hljs-number">0</span> slots) master
   replicates e0701859f47b7c054b164b4cf5102798192f2500
[OK] <span class="hljs-keyword">All</span> nodes agree about slots configuration.
>>> <span class="hljs-keyword">Check</span> <span class="hljs-keyword">for</span> <span class="hljs-keyword">open</span> slots...
>>> <span class="hljs-keyword">Check</span> slots coverage...
[OK] <span class="hljs-keyword">All</span> <span class="hljs-number">16384</span> slots covered.
</span></code>

Finding nodes and redirects

客户端可以连接集群中的任意节点执行查询（包括replica），但是这个节点未必有需要的数据，因此客户端需要负责去定位key所在的节点，然后再重定向到正确的节点。而这可以做到的原因，是因为通过hash计算可以得到slot，而每个实例的slot是固定的。

例如:

<code class=" hljs css">$ <span class="hljs-tag">redis-cli</span> <span class="hljs-tag">-c</span> <span class="hljs-tag">-h</span> <span class="hljs-tag">localhost</span> <span class="hljs-tag">-p</span> 30001
<span class="hljs-tag">localhost</span><span class="hljs-pseudo">:30001</span>> <span class="hljs-tag">SET</span> <span class="hljs-tag">hello</span> <span class="hljs-tag">world</span>
<span class="hljs-tag">OK</span>
<span class="hljs-tag">localhost</span><span class="hljs-pseudo">:30001</span>> <span class="hljs-tag">SET</span> <span class="hljs-tag">foo</span> <span class="hljs-tag">bar</span>
<span class="hljs-tag">-</span>> <span class="hljs-tag">Redirected</span> <span class="hljs-tag">to</span> <span class="hljs-tag">slot</span> <span class="hljs-attr_selector">[12182]</span> <span class="hljs-tag">located</span> <span class="hljs-tag">at</span> 127<span class="hljs-class">.0</span><span class="hljs-class">.0</span><span class="hljs-class">.1</span><span class="hljs-pseudo">:30003</span>
<span class="hljs-tag">OK</span>
127<span class="hljs-class">.0</span><span class="hljs-class">.0</span><span class="hljs-class">.1</span><span class="hljs-pseudo">:30003</span>> <span class="hljs-tag">GET</span> <span class="hljs-tag">foo</span>
"<span class="hljs-tag">bar</span>"
127<span class="hljs-class">.0</span><span class="hljs-class">.0</span><span class="hljs-class">.1</span><span class="hljs-pseudo">:30003</span>> <span class="hljs-tag">GET</span> <span class="hljs-tag">hello</span>
<span class="hljs-tag">-</span>> <span class="hljs-tag">Redirected</span> <span class="hljs-tag">to</span> <span class="hljs-tag">slot</span> <span class="hljs-attr_selector">[866]</span> <span class="hljs-tag">located</span> <span class="hljs-tag">at</span> 127<span class="hljs-class">.0</span><span class="hljs-class">.0</span><span class="hljs-class">.1</span><span class="hljs-pseudo">:30001</span>
"<span class="hljs-tag">world</span>"</code>

foo和hello的slot#为12182和866, 而3个master的slot分配如下：
127.0.0.1:30001 -> 0 到 5460
127.0.0.1:30002 -> 5461 到 10922
127.0.0.1:30003 -> 10923 到 16383

同时我们也可以验证一下前面讲的tag的功效:

<code class=" hljs ruby"><span class="hljs-symbol">localhost:</span><span class="hljs-number">30001</span>> <span class="hljs-constant">SADD</span> {user123}<span class="hljs-symbol">:friends</span><span class="hljs-symbol">:usa</span> <span class="hljs-string">"John"</span> <span class="hljs-string">"Bob"</span>
-> <span class="hljs-constant">Redirected</span> to slot [<span class="hljs-number">13438</span>] located at <span class="hljs-number">127.0</span>.<span class="hljs-number">0</span>.<span class="hljs-number">1</span><span class="hljs-symbol">:</span><span class="hljs-number">30003</span>
(integer) <span class="hljs-number">2</span>
<span class="hljs-number">127.0</span>.<span class="hljs-number">0</span>.<span class="hljs-number">1</span><span class="hljs-symbol">:</span><span class="hljs-number">30003</span>> <span class="hljs-constant">SADD</span> {user123}<span class="hljs-symbol">:friends</span><span class="hljs-symbol">:brazil</span> <span class="hljs-string">"Max"</span> <span class="hljs-string">"Hugo"</span>
(integer) <span class="hljs-number">2</span>
<span class="hljs-number">127.0</span>.<span class="hljs-number">0</span>.<span class="hljs-number">1</span><span class="hljs-symbol">:</span><span class="hljs-number">30003</span>> <span class="hljs-constant">SUNION</span> {user123}<span class="hljs-symbol">:all_friends</span> {user123}<span class="hljs-symbol">:friends</span><span class="hljs-symbol">:usa</span> {user123}<span class="hljs-symbol">:friends</span><span class="hljs-symbol">:brazil</span>
<span class="hljs-number">1</span>) <span class="hljs-string">"Bob"</span>
<span class="hljs-number">2</span>) <span class="hljs-string">"Max"</span>
<span class="hljs-number">3</span>) <span class="hljs-string">"Hugo"</span>
<span class="hljs-number">4</span>) <span class="hljs-string">"John"</span>
</code>

Configuration

Redis运行在Cluster模式需要设置相关指令，否则以单实例模式运行，相关指令如下:

cluster-enabled 缺省为no, 不启动cluster模式
cluster-config-file cluster.conf 配置文件
cluster-node-timeout 2000 多长时间认为节点失效
cluster-slave-validity-factor 10 为防止网络临时故障，cluster-node-timeout x cluster-slave-validity-factor 时间内，slave不会升级为master
cluster-migration-barrier 1 从其他master借用slave的门限
cluster-require-full-coverage yes 如果master失效时，没有slave可以提升，则数据会丢失。这时是否设置为整个cluster失效，还是让cluster继续服务

Different Redis Cluster architectures

前面我们建议每一个master都配至少一个slave。
架构1：
如果每一个master都配一个slave，缺点是如果master失效，slave升级为master，如果这个节点发生二次故障，则数据丢失
架构2：
如果每一个master都配2个或以上slave，缺点是不经济
架构3：
类似于磁盘阵列的全局热备盘，可以为少量节点配置2个或以上slave，而其余节点配置1个slave。当后者master失效后，可以借用前者的slave，以防止二次故障

Cluster administration

接下来我们讲如何用Redis自己的命令来创建和管理Cluster

Creating a cluster

接下来，我们创建3个master。

<code class=" hljs lasso">$ redis<span class="hljs-attribute">-server</span> <span class="hljs-subst">--</span>port <span class="hljs-number">5000</span> <span class="hljs-subst">--</span>cluster<span class="hljs-attribute">-enabled</span> yes <span class="hljs-subst">--</span>cluster<span class="hljs-attribute">-config</span><span class="hljs-attribute">-file</span> nodes<span class="hljs-subst">-</span><span class="hljs-number">5000.</span>conf <span class="hljs-subst">--</span>cluster<span class="hljs-attribute">-node</span><span class="hljs-attribute">-timeout</span> <span class="hljs-number">2000</span> <span class="hljs-subst">--</span>cluster<span class="hljs-attribute">-slave</span><span class="hljs-attribute">-validity</span><span class="hljs-attribute">-factor</span> <span class="hljs-number">10</span> <span class="hljs-subst">--</span>cluster<span class="hljs-attribute">-migration</span><span class="hljs-attribute">-barrier</span> <span class="hljs-number">1</span> <span class="hljs-subst">--</span>cluster<span class="hljs-attribute">-require</span><span class="hljs-attribute">-full</span><span class="hljs-attribute">-coverage</span> yes <span class="hljs-subst">--</span>dbfilename dump<span class="hljs-subst">-</span><span class="hljs-number">5000.</span>rdb  <span class="hljs-subst">--</span>daemonize yes

$ redis<span class="hljs-attribute">-server</span> <span class="hljs-subst">--</span>port <span class="hljs-number">5001</span> <span class="hljs-subst">--</span>cluster<span class="hljs-attribute">-enabled</span> yes <span class="hljs-subst">--</span>cluster<span class="hljs-attribute">-config</span><span class="hljs-attribute">-file</span> nodes<span class="hljs-subst">-</span><span class="hljs-number">5001.</span>conf <span class="hljs-subst">--</span>cluster<span class="hljs-attribute">-node</span><span class="hljs-attribute">-timeout</span> <span class="hljs-number">2000</span> <span class="hljs-subst">--</span>cluster<span class="hljs-attribute">-slave</span><span class="hljs-attribute">-validity</span><span class="hljs-attribute">-factor</span> <span class="hljs-number">10</span> <span class="hljs-subst">--</span>cluster<span class="hljs-attribute">-migration</span><span class="hljs-attribute">-barrier</span> <span class="hljs-number">1</span> <span class="hljs-subst">--</span>cluster<span class="hljs-attribute">-require</span><span class="hljs-attribute">-full</span><span class="hljs-attribute">-coverage</span> yes <span class="hljs-subst">--</span>dbfilename dump<span class="hljs-subst">-</span><span class="hljs-number">5001.</span>rdb <span class="hljs-subst">--</span>daemonize yes

$ redis<span class="hljs-attribute">-server</span> <span class="hljs-subst">--</span>port <span class="hljs-number">5002</span> <span class="hljs-subst">--</span>cluster<span class="hljs-attribute">-enabled</span> yes <span class="hljs-subst">--</span>cluster<span class="hljs-attribute">-config</span><span class="hljs-attribute">-file</span> nodes<span class="hljs-subst">-</span><span class="hljs-number">5002.</span>conf <span class="hljs-subst">--</span>cluster<span class="hljs-attribute">-node</span><span class="hljs-attribute">-timeout</span> <span class="hljs-number">2000</span> <span class="hljs-subst">--</span>cluster<span class="hljs-attribute">-slave</span><span class="hljs-attribute">-validity</span><span class="hljs-attribute">-factor</span> <span class="hljs-number">10</span> <span class="hljs-subst">--</span>cluster<span class="hljs-attribute">-migration</span><span class="hljs-attribute">-barrier</span> <span class="hljs-number">1</span> <span class="hljs-subst">--</span>cluster<span class="hljs-attribute">-require</span><span class="hljs-attribute">-full</span><span class="hljs-attribute">-coverage</span> yes <span class="hljs-subst">--</span>dbfilename dump<span class="hljs-subst">-</span><span class="hljs-number">5002.</span>rdb <span class="hljs-subst">--</span>daemonize yes

$ ps <span class="hljs-attribute">-ef</span><span class="hljs-subst">|</span>grep redis<span class="hljs-attribute">-server</span>
redis     <span class="hljs-number">3467</span>     <span class="hljs-number">1</span>  <span class="hljs-number">0</span> <span class="hljs-number">11</span>:<span class="hljs-number">23</span> <span class="hljs-subst">?</span>        <span class="hljs-number">00</span>:<span class="hljs-number">00</span>:<span class="hljs-number">00</span> redis<span class="hljs-attribute">-server</span> <span class="hljs-subst">*</span>:<span class="hljs-number">5000</span> <span class="hljs-preprocessor">[</span>cluster<span class="hljs-preprocessor">]</span><span class="hljs-markup">                                                                    
redis     3476     1  0 11:23 ?        00:00:00 redis-server *:5001 </span><span class="hljs-preprocessor">[</span>cluster<span class="hljs-preprocessor">]</span><span class="hljs-markup">                                                                    
redis     3480     1  0 11:23 ?        00:00:00 redis-server *:5002 </span><span class="hljs-preprocessor">[</span>cluster<span class="hljs-preprocessor">]</span><span class="hljs-markup"> </span></code>

cluster还没有运行，我们用CLUSTER INFO检查一下状态:

<code class=" hljs css">$ <span class="hljs-tag">redis-cli</span> <span class="hljs-tag">-c</span> <span class="hljs-tag">-p</span> 5000
127<span class="hljs-class">.0</span><span class="hljs-class">.0</span><span class="hljs-class">.1</span><span class="hljs-pseudo">:5000</span>> <span class="hljs-tag">CLUSTER</span> <span class="hljs-tag">INFO</span>
<span class="hljs-tag">cluster_state</span><span class="hljs-pseudo">:fail</span>
<span class="hljs-tag">cluster_slots_assigned</span><span class="hljs-pseudo">:0</span>
<span class="hljs-tag">cluster_slots_ok</span><span class="hljs-pseudo">:0</span>
<span class="hljs-tag">cluster_slots_pfail</span><span class="hljs-pseudo">:0</span>
<span class="hljs-tag">cluster_slots_fail</span><span class="hljs-pseudo">:0</span>
<span class="hljs-tag">cluster_known_nodes</span><span class="hljs-pseudo">:1</span>
<span class="hljs-tag">cluster_size</span><span class="hljs-pseudo">:0</span>
<span class="hljs-tag">cluster_current_epoch</span><span class="hljs-pseudo">:0</span>
<span class="hljs-tag">cluster_my_epoch</span><span class="hljs-pseudo">:0</span>
<span class="hljs-tag">cluster_stats_messages_sent</span><span class="hljs-pseudo">:0</span>
<span class="hljs-tag">cluster_stats_messages_received</span><span class="hljs-pseudo">:0</span>
127<span class="hljs-class">.0</span><span class="hljs-class">.0</span><span class="hljs-class">.1</span><span class="hljs-pseudo">:5000</span>> <span class="hljs-tag">SET</span> <span class="hljs-tag">foo</span> <span class="hljs-tag">bar</span>
(<span class="hljs-tag">error</span>) <span class="hljs-tag">CLUSTERDOWN</span> <span class="hljs-tag">The</span> <span class="hljs-tag">cluster</span> <span class="hljs-tag">is</span> <span class="hljs-tag">down</span></code>

输出显示, cluster只知道一个节点，slot尚未分配，cluster状态是down，这时节点不能接受请求

接下来，分配slot，cluster-require-full-coverage设为yes，表示必须分配所有的slot后，cluster才能接受请求nces:

<code class=" hljs ruby">[redis<span class="hljs-variable">@tt12c</span> test]<span class="hljs-variable">$ </span>redis-cli -c -p <span class="hljs-number">5000</span> <span class="hljs-constant">CLUSTER</span> <span class="hljs-constant">ADDSLOTS</span> {<span class="hljs-number">0</span>..<span class="hljs-number">5460</span>}
<span class="hljs-constant">OK</span>
[redis<span class="hljs-variable">@tt12c</span> test]<span class="hljs-variable">$ </span>redis-cli -c -p <span class="hljs-number">5001</span> <span class="hljs-constant">CLUSTER</span> <span class="hljs-constant">ADDSLOTS</span> {<span class="hljs-number">5461</span>..<span class="hljs-number">10922</span>}
<span class="hljs-constant">OK</span>
[redis<span class="hljs-variable">@tt12c</span> test]<span class="hljs-variable">$ </span>redis-cli -c -p <span class="hljs-number">5002</span> <span class="hljs-constant">CLUSTER</span> <span class="hljs-constant">ADDSLOTS</span> {<span class="hljs-number">10923</span>..<span class="hljs-number">16383</span>}
<span class="hljs-constant">OK</span></code>

此时，cluster仍未就绪，节点之间还相互不知道。

Redis Cluster中有一个概念叫configuration epoch，是一个数字，相当于逻辑时钟。

这个数字在多个节点需要就一些事情达成一致时使用，如failover, resharding。

cluster初始化时，configuration epoch设置为0，也可以在启动cluster时设置一次，这时唯一需要手工设置的地方，其后，Redis会自动管理。

<code class=" hljs ruby">[redis<span class="hljs-variable">@tt12c</span> test]<span class="hljs-variable">$ </span>redis-cli -c -p <span class="hljs-number">5000</span> <span class="hljs-constant">CLUSTER</span> <span class="hljs-constant">SET</span>-<span class="hljs-constant">CONFIG</span>-<span class="hljs-constant">EPOCH</span> <span class="hljs-number">1</span>
<span class="hljs-constant">OK</span>
[redis<span class="hljs-variable">@tt12c</span> test]<span class="hljs-variable">$ </span>redis-cli -c -p <span class="hljs-number">5001</span> <span class="hljs-constant">CLUSTER</span> <span class="hljs-constant">SET</span>-<span class="hljs-constant">CONFIG</span>-<span class="hljs-constant">EPOCH</span> <span class="hljs-number">2</span>
<span class="hljs-constant">OK</span>
[redis<span class="hljs-variable">@tt12c</span> test]<span class="hljs-variable">$ </span>redis-cli -c -p <span class="hljs-number">5002</span> <span class="hljs-constant">CLUSTER</span> <span class="hljs-constant">SET</span>-<span class="hljs-constant">CONFIG</span>-<span class="hljs-constant">EPOCH</span> <span class="hljs-number">3</span></code>

然后，我们让节点之间能相互知道，没必要让每一个节点都轮番执行以下命令，Redis会自动形成full mesh网络。

<code class=" hljs ruby">[redis<span class="hljs-variable">@tt12c</span> test]<span class="hljs-variable">$ </span>redis-cli -c -p <span class="hljs-number">5000</span> <span class="hljs-constant">CLUSTER</span> <span class="hljs-constant">MEET</span> <span class="hljs-number">127.0</span>.<span class="hljs-number">0</span>.<span class="hljs-number">1</span> <span class="hljs-number">5001</span>
<span class="hljs-constant">OK</span>
[redis<span class="hljs-variable">@tt12c</span> test]<span class="hljs-variable">$ </span>redis-cli -c -p <span class="hljs-number">5000</span> <span class="hljs-constant">CLUSTER</span> <span class="hljs-constant">MEET</span> <span class="hljs-number">127.0</span>.<span class="hljs-number">0</span>.<span class="hljs-number">1</span> <span class="hljs-number">5002</span>
<span class="hljs-constant">OK</span></code>

再次运行CLUSTER INFO，这时cluster已经就绪了

<code class=" hljs css">$ <span class="hljs-tag">redis-cli</span> <span class="hljs-tag">-c</span> <span class="hljs-tag">-p</span> 5000
<span class="hljs-attr_selector">[redis@tt12c test]</span>$ <span class="hljs-tag">redis-cli</span> <span class="hljs-tag">-c</span> <span class="hljs-tag">-p</span> 5000
127<span class="hljs-class">.0</span><span class="hljs-class">.0</span><span class="hljs-class">.1</span><span class="hljs-pseudo">:5000</span>> <span class="hljs-tag">CLUSTER</span> <span class="hljs-tag">INFO</span>
<span class="hljs-tag">cluster_state</span><span class="hljs-pseudo">:ok</span>
<span class="hljs-tag">cluster_slots_assigned</span><span class="hljs-pseudo">:16384</span>
<span class="hljs-tag">cluster_slots_ok</span><span class="hljs-pseudo">:16384</span>
<span class="hljs-tag">cluster_slots_pfail</span><span class="hljs-pseudo">:0</span>
<span class="hljs-tag">cluster_slots_fail</span><span class="hljs-pseudo">:0</span>
<span class="hljs-tag">cluster_known_nodes</span><span class="hljs-pseudo">:3</span>
<span class="hljs-tag">cluster_size</span><span class="hljs-pseudo">:3</span>
<span class="hljs-tag">cluster_current_epoch</span><span class="hljs-pseudo">:3</span>
<span class="hljs-tag">cluster_my_epoch</span><span class="hljs-pseudo">:1</span>
<span class="hljs-tag">cluster_stats_messages_sent</span><span class="hljs-pseudo">:514</span>
<span class="hljs-tag">cluster_stats_messages_received</span><span class="hljs-pseudo">:514</span></code>

Adding slaves/replicas

目前，还没有slave，因此也没有复制。

salve可以通过以下步骤加入cluster:
1. 在cluster模式下创建新实例
2. 用CLUSTER MEET命令加入cluster
3. 用CLUSTER NODES得到master的node ID
4. 用CLUSTER REPLICATE建立复制关系

示例如下：

<code class=" hljs css"><span class="hljs-attr_selector">[redis@tt12c test]</span>$ <span class="hljs-tag">redis-server</span> <span class="hljs-tag">--port</span> 5003 <span class="hljs-tag">--cluster-enabled</span> <span class="hljs-tag">yes</span> <span class="hljs-tag">--cluster-config-file</span> <span class="hljs-tag">nodes-5003</span><span class="hljs-class">.conf</span> <span class="hljs-tag">--cluster-node-timeout</span> 2000 <span class="hljs-tag">--cluster-slave-validity-factor</span> 10 <span class="hljs-tag">--cluster-migration-barrier</span> 1 <span class="hljs-tag">--cluster-require-full-coverage</span> <span class="hljs-tag">yes</span> <span class="hljs-tag">--dbfilename</span> <span class="hljs-tag">dump-5003</span><span class="hljs-class">.rdb</span> <span class="hljs-tag">--daemonize</span> <span class="hljs-tag">yes</span>
<span class="hljs-attr_selector">[redis@tt12c test]</span>$ <span class="hljs-tag">redis-cli</span> <span class="hljs-tag">-c</span> <span class="hljs-tag">-p</span> 5003 <span class="hljs-tag">CLUSTER</span> <span class="hljs-tag">MEET</span> 127<span class="hljs-class">.0</span><span class="hljs-class">.0</span><span class="hljs-class">.1</span> 5000
<span class="hljs-tag">OK</span>
<span class="hljs-attr_selector">[redis@tt12c test]</span>$ <span class="hljs-tag">redis-cli</span> <span class="hljs-tag">-c</span> <span class="hljs-tag">-p</span> 5003 <span class="hljs-tag">CLUSTER</span> <span class="hljs-tag">NODES</span>
98177663<span class="hljs-tag">af8067766ab855de02ad37d3f9f1c691</span> 127<span class="hljs-class">.0</span><span class="hljs-class">.0</span><span class="hljs-class">.1</span><span class="hljs-pseudo">:5001</span> <span class="hljs-tag">master</span> <span class="hljs-tag">-</span> 0 1462765861356 2 <span class="hljs-tag">connected</span> 5461<span class="hljs-tag">-10922</span>
<span class="hljs-tag">b343c84935a93d409b87da13368f08c90bec0171</span> 127<span class="hljs-class">.0</span><span class="hljs-class">.0</span><span class="hljs-class">.1</span><span class="hljs-pseudo">:5000</span> <span class="hljs-tag">master</span> <span class="hljs-tag">-</span> 0 1462765861962 1 <span class="hljs-tag">connected</span> 0<span class="hljs-tag">-5460</span> <<span class="hljs-tag">-</span> <span class="hljs-tag">master</span>的<span class="hljs-tag">node</span> <span class="hljs-tag">ID</span>
47<span class="hljs-tag">c8494ecbd9a130878449827719834634d1b533</span> 127<span class="hljs-class">.0</span><span class="hljs-class">.0</span><span class="hljs-class">.1</span><span class="hljs-pseudo">:5002</span> <span class="hljs-tag">master</span> <span class="hljs-tag">-</span> 0 1462765861962 3 <span class="hljs-tag">connected</span> 10923<span class="hljs-tag">-16383</span>
<span class="hljs-tag">cbcfc9234b8e664890b71efe7f28325d0727d622</span> 127<span class="hljs-class">.0</span><span class="hljs-class">.0</span><span class="hljs-class">.1</span><span class="hljs-pseudo">:5003</span> <span class="hljs-tag">myself</span>,<span class="hljs-tag">master</span> <span class="hljs-tag">-</span> 0 0 0 <span class="hljs-tag">connected</span> <<span class="hljs-tag">-</span> 暂为<span class="hljs-tag">master</span>
<span class="hljs-attr_selector">[redis@tt12c test]</span>$ <span class="hljs-tag">redis-cli</span> <span class="hljs-tag">-c</span> <span class="hljs-tag">-p</span> 5003 <span class="hljs-tag">CLUSTER</span> <span class="hljs-tag">REPLICATE</span> <span class="hljs-tag">b343c84935a93d409b87da13368f08c90bec0171</span>
<span class="hljs-tag">OK</span>
<span class="hljs-attr_selector">[redis@tt12c test]</span>$ <span class="hljs-tag">redis-cli</span> <span class="hljs-tag">-c</span> <span class="hljs-tag">-p</span> 5003 <span class="hljs-tag">CLUSTER</span> <span class="hljs-tag">NODES</span>
98177663<span class="hljs-tag">af8067766ab855de02ad37d3f9f1c691</span> 127<span class="hljs-class">.0</span><span class="hljs-class">.0</span><span class="hljs-class">.1</span><span class="hljs-pseudo">:5001</span> <span class="hljs-tag">master</span> <span class="hljs-tag">-</span> 0 1462766183413 2 <span class="hljs-tag">connected</span> 5461<span class="hljs-tag">-10922</span>
<span class="hljs-tag">b343c84935a93d409b87da13368f08c90bec0171</span> 127<span class="hljs-class">.0</span><span class="hljs-class">.0</span><span class="hljs-class">.1</span><span class="hljs-pseudo">:5000</span> <span class="hljs-tag">master</span> <span class="hljs-tag">-</span> 0 1462766183413 1 <span class="hljs-tag">connected</span> 0<span class="hljs-tag">-5460</span>
47<span class="hljs-tag">c8494ecbd9a130878449827719834634d1b533</span> 127<span class="hljs-class">.0</span><span class="hljs-class">.0</span><span class="hljs-class">.1</span><span class="hljs-pseudo">:5002</span> <span class="hljs-tag">master</span> <span class="hljs-tag">-</span> 0 1462766183413 3 <span class="hljs-tag">connected</span> 10923<span class="hljs-tag">-16383</span>
<span class="hljs-tag">cbcfc9234b8e664890b71efe7f28325d0727d622</span> 127<span class="hljs-class">.0</span><span class="hljs-class">.0</span><span class="hljs-class">.1</span><span class="hljs-pseudo">:5003</span> <span class="hljs-tag">myself</span>,<span class="hljs-tag">slave</span> <span class="hljs-tag">b343c84935a93d409b87da13368f08c90bec0171</span> 0 0 0 <span class="hljs-tag">connected</span> <<span class="hljs-tag">-</span> 变为<span class="hljs-tag">slave</span>
</code>

node ID使用/dev/urandom产生的。

Scaling reads using slave nodes

可以用READONLY命令将slave设置只读模式来扩展读，这时，它只会转发自己无法处理的命令，缺点是可能读到旧数据。
可以用READWRITE退出只读模式。

如果没有启动只读模式，所有的请求都会由master处理，可能成为瓶颈，而slave处于空闲状态。

Adding nodes

当cluster新增一个节点时，状态为master，且没有slot分配。此节点接受请求后都将转发到其它节点。接下来我们看resharding是如何做的。步骤如下：
1. 在cluster模式下创建新实例
2. 用CLUSTER MEET命令加入cluster
3. 用CLUSTER NODES得到新节点和目标节点的node ID
4. 用CLUSTER SETSLOT 和 MIGRATE命令重新分布hash slot

示例如下：

<code class=" hljs mel">[redis<span class="hljs-variable">@tt12c</span> test]$ redis-server --port <span class="hljs-number">6000</span> --<span class="hljs-keyword">cluster</span>-enabled yes --<span class="hljs-keyword">cluster</span>-config-<span class="hljs-keyword">file</span> nodes-<span class="hljs-number">6000.</span>conf --<span class="hljs-keyword">cluster</span>-node-timeout <span class="hljs-number">2000</span> --<span class="hljs-keyword">cluster</span>-slave-validity-factor <span class="hljs-number">10</span> --<span class="hljs-keyword">cluster</span>-migration-barrier <span class="hljs-number">1</span> --<span class="hljs-keyword">cluster</span>-require-full-coverage yes --dbfilename dump-<span class="hljs-number">6000.</span>rdb --daemonize yes
[redis<span class="hljs-variable">@tt12c</span> test]$ redis-cli -c -p <span class="hljs-number">6000</span> CLUSTER MEET <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span> <span class="hljs-number">5000</span>
OK
[redis<span class="hljs-variable">@tt12c</span> test]$ redis-cli -c -p <span class="hljs-number">6000</span> CLUSTER NODES
<span class="hljs-number">0e14</span>d30c8d3f780fd3bb9b4ea3955f520309248a <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">6000</span> myself,master - <span class="hljs-number">0</span> <span class="hljs-number">0</span> <span class="hljs-number">0</span> connected
b343c84935a93d409b87da13368f08c90bec0171 <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">5000</span> master - <span class="hljs-number">0</span> <span class="hljs-number">1462766842370</span> <span class="hljs-number">1</span> connected <span class="hljs-number">0</span>-<span class="hljs-number">5460</span>
<span class="hljs-number">47</span>c8494ecbd9a130878449827719834634d1b533 <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">5002</span> master - <span class="hljs-number">0</span> <span class="hljs-number">1462766841664</span> <span class="hljs-number">3</span> connected <span class="hljs-number">10923</span>-<span class="hljs-number">16383</span>
cbcfc9234b8e664890b71efe7f28325d0727d622 <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">5003</span> slave b343c84935a93d409b87da13368f08c90bec0171 <span class="hljs-number">0</span> <span class="hljs-number">1462766841765</span> <span class="hljs-number">1</span> connected
<span class="hljs-number">98177663</span>af8067766ab855de02ad37d3f9f1c691 <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">5001</span> master - <span class="hljs-number">0</span> <span class="hljs-number">1462766842370</span> <span class="hljs-number">2</span> connected <span class="hljs-number">5461</span>-<span class="hljs-number">10922</span>
[redis<span class="hljs-variable">@tt12c</span> test]$ redis-cli -c -p <span class="hljs-number">6000</span> SET book <span class="hljs-string">"redis essentials"</span>
OK
</code>

Redis Cluster每次只能迁移(reshard)一个slot，步骤如下:
1. 目标节点进入import状态
2. 源节点进入export状态
3. 迁移key
4. 通知所有的master

resharding的命令是CLUSTER SETSLOT，包括IMPORTING, MIGRATING, NODE, 和 STABLE子命令，简介如下：
CLUSTER SETSLOT IMPORTING : 在目标节点上执行，进入import状态
CLUSTER SETSLOT MIGRATING : 在源节点上执行，进入migrate状态
CLUSTER SETSLOT NODE :将slot与节点关联，目标和源节点上都必须执行，建议在所有master上执行
CLUSTER SETSLOT STABLE: 清除importing和migrating的状态，可用于回退操作

示例如下：

<code class=" hljs ruby">[redis<span class="hljs-variable">@tt12c</span> test]<span class="hljs-variable">$ </span>redis-cli -c -p <span class="hljs-number">6000</span> 
<span class="hljs-number">127.0</span>.<span class="hljs-number">0</span>.<span class="hljs-number">1</span><span class="hljs-symbol">:</span><span class="hljs-number">6000</span>> <span class="hljs-constant">SET</span> book <span class="hljs-string">"redis essentials"</span>
-> <span class="hljs-constant">Redirected</span> to slot [<span class="hljs-number">1337</span>] located at <span class="hljs-number">127.0</span>.<span class="hljs-number">0</span>.<span class="hljs-number">1</span><span class="hljs-symbol">:</span><span class="hljs-number">5000</span>
<span class="hljs-constant">OK</span>
<span class="hljs-number">127.0</span>.<span class="hljs-number">0</span>.<span class="hljs-number">1</span><span class="hljs-symbol">:</span><span class="hljs-number">5000</span>> exit

[redis<span class="hljs-variable">@tt12c</span> test]<span class="hljs-variable">$ </span>redis-cli -c -p <span class="hljs-number">6000</span> <span class="hljs-constant">CLUSTER</span> <span class="hljs-constant">SETSLOT</span> <span class="hljs-number">1337</span> <span class="hljs-constant">IMPORTING</span> 0e14d30c8d3f780fd3bb9b4ea3955f520309248a
<span class="hljs-constant">OK</span> <- 目标进入import状态
[redis<span class="hljs-variable">@tt12c</span> test]<span class="hljs-variable">$ </span>redis-cli -c -p <span class="hljs-number">5000</span> <span class="hljs-constant">CLUSTER</span> <span class="hljs-constant">SETSLOT</span> <span class="hljs-number">1337</span> <span class="hljs-constant">MIGRATING</span> b343c84935a93d409b87da13368f08c90bec0171
<span class="hljs-constant">OK</span> <- 源进入migrate状态

[redis<span class="hljs-variable">@tt12c</span> test]<span class="hljs-variable">$ </span>redis-cli -c -p <span class="hljs-number">5000</span>
<span class="hljs-number">127.0</span>.<span class="hljs-number">0</span>.<span class="hljs-number">1</span><span class="hljs-symbol">:</span><span class="hljs-number">5000</span>> <span class="hljs-constant">CLUSTER</span> <span class="hljs-constant">COUNTKEYSINSLOT</span> <span class="hljs-number">1337</span> <- slot中的key数量
(integer) <span class="hljs-number">1</span>
<span class="hljs-number">127.0</span>.<span class="hljs-number">0</span>.<span class="hljs-number">1</span><span class="hljs-symbol">:</span><span class="hljs-number">5000</span>> <span class="hljs-constant">CLUSTER</span> <span class="hljs-constant">GETKEYSINSLOT</span> <span class="hljs-number">1337</span> <span class="hljs-number">1</span> <- slot中的key列表 
<span class="hljs-number">1</span>) <span class="hljs-string">"book"</span>
<span class="hljs-number">127.0</span>.<span class="hljs-number">0</span>.<span class="hljs-number">1</span><span class="hljs-symbol">:</span><span class="hljs-number">5000</span>> <span class="hljs-constant">MIGRATE</span> <span class="hljs-number">127.0</span>.<span class="hljs-number">0</span>.<span class="hljs-number">1</span> <span class="hljs-number">6000</span> book <span class="hljs-number">0</span> <span class="hljs-number">2000</span> <- 迁移一个<span class="hljs-symbol">key:</span> book
<span class="hljs-constant">OK</span>
<span class="hljs-number">127.0</span>.<span class="hljs-number">0</span>.<span class="hljs-number">1</span><span class="hljs-symbol">:</span><span class="hljs-number">5000</span>> help migrate

  <span class="hljs-constant">MIGRATE</span> host port key destination-db timeout [<span class="hljs-constant">COPY</span>] [<span class="hljs-constant">REPLACE</span>]
  <span class="hljs-symbol">summary:</span> <span class="hljs-constant">Atomically</span> transfer a key from a <span class="hljs-constant">Redis</span> instance to another one.
  <span class="hljs-symbol">since:</span> <span class="hljs-number">2.6</span>.<span class="hljs-number">0</span>
  <span class="hljs-symbol">group:</span> generic

<span class="hljs-comment"># 通知所有的节点</span>
[redis<span class="hljs-variable">@tt12c</span> test]<span class="hljs-variable">$ </span>redis-cli -c -p <span class="hljs-number">5000</span> <span class="hljs-constant">CLUSTER</span> <span class="hljs-constant">SETSLOT</span> <span class="hljs-number">1337</span> <span class="hljs-constant">NODE</span> b343c84935a93d409b87da13368f08c90bec0171
<span class="hljs-constant">OK</span>
[redis<span class="hljs-variable">@tt12c</span> test]<span class="hljs-variable">$ </span>redis-cli -c -p <span class="hljs-number">5001</span> <span class="hljs-constant">CLUSTER</span> <span class="hljs-constant">SETSLOT</span> <span class="hljs-number">1337</span> <span class="hljs-constant">NODE</span> <span class="hljs-number">98177663</span>af8067766ab855de02ad37d3f9f1c691
<span class="hljs-constant">OK</span>
[redis<span class="hljs-variable">@tt12c</span> test]<span class="hljs-variable">$ </span>redis-cli -c -p <span class="hljs-number">5002</span> <span class="hljs-constant">CLUSTER</span> <span class="hljs-constant">SETSLOT</span> <span class="hljs-number">1337</span> <span class="hljs-constant">NODE</span> <span class="hljs-number">47</span>c8494ecbd9a130878449827719834634d1b533
<span class="hljs-constant">OK</span>
[redis<span class="hljs-variable">@tt12c</span> test]<span class="hljs-variable">$ </span>redis-cli -c -p <span class="hljs-number">6000</span> <span class="hljs-constant">CLUSTER</span> <span class="hljs-constant">SETSLOT</span> <span class="hljs-number">1337</span> <span class="hljs-constant">NODE</span> cbcfc9234b8e664890b71efe7f28325d0727d622
<span class="hljs-constant">OK</span>
</code>

新的slot分配可以用CLUSTER NODES验证：

<code class=" hljs css"><span class="hljs-attr_selector">[redis@tt12c test]</span>$ <span class="hljs-tag">redis-cli</span> <span class="hljs-tag">-c</span> <span class="hljs-tag">-p</span> 6000 <span class="hljs-tag">CLUSTER</span> <span class="hljs-tag">NODES</span>
0<span class="hljs-tag">e14d30c8d3f780fd3bb9b4ea3955f520309248a</span> 127<span class="hljs-class">.0</span><span class="hljs-class">.0</span><span class="hljs-class">.1</span><span class="hljs-pseudo">:6000</span> <span class="hljs-tag">myself</span>,<span class="hljs-tag">master</span> <span class="hljs-tag">-</span> 0 0 0 <span class="hljs-tag">connected</span> <span class="hljs-attr_selector">[1337-<-0e14d30c8d3f780fd3bb9b4ea3955f520309248a]</span>
<span class="hljs-tag">b343c84935a93d409b87da13368f08c90bec0171</span> 127<span class="hljs-class">.0</span><span class="hljs-class">.0</span><span class="hljs-class">.1</span><span class="hljs-pseudo">:5000</span> <span class="hljs-tag">master</span> <span class="hljs-tag">-</span> 0 1462768263599 1 <span class="hljs-tag">connected</span> 0<span class="hljs-tag">-1336</span> 1338<span class="hljs-tag">-5460</span>
47<span class="hljs-tag">c8494ecbd9a130878449827719834634d1b533</span> 127<span class="hljs-class">.0</span><span class="hljs-class">.0</span><span class="hljs-class">.1</span><span class="hljs-pseudo">:5002</span> <span class="hljs-tag">master</span> <span class="hljs-tag">-</span> 0 1462768263599 3 <span class="hljs-tag">connected</span> 10923<span class="hljs-tag">-16383</span>
<span class="hljs-tag">cbcfc9234b8e664890b71efe7f28325d0727d622</span> 127<span class="hljs-class">.0</span><span class="hljs-class">.0</span><span class="hljs-class">.1</span><span class="hljs-pseudo">:5003</span> <span class="hljs-tag">slave</span> <span class="hljs-tag">b343c84935a93d409b87da13368f08c90bec0171</span> 0 1462768263599 1 <span class="hljs-tag">connected</span> 1337
98177663<span class="hljs-tag">af8067766ab855de02ad37d3f9f1c691</span> 127<span class="hljs-class">.0</span><span class="hljs-class">.0</span><span class="hljs-class">.1</span><span class="hljs-pseudo">:5001</span> <span class="hljs-tag">master</span> <span class="hljs-tag">-</span> 0 1462768263599 2 <span class="hljs-tag">connected</span> 5461<span class="hljs-tag">-10922</span>
</code>

Removing nodes

删除一个节点需要resharding，然后所有节点必须从节点列表中取出此节点。

在hash slot做了resharding后，在所有master节点上在60秒内执行CLUSTER FORGET 就可以删除此节点。

Redis Cluster administration using the redis-trib tool

redis-trib是官方的Cluster管理工具，是Ruby脚本，就是封装了之前介绍的命令，还不太成熟，但比命令行还是简单多了。

<code class=" hljs haml">[redis@tt12c ~]$ cd redis-3.0.7/
[redis@tt12c redis-3.0.7]$ cd src
[redis@tt12c src]$ ./redis-trib.rb 
Usage: redis-trib <command> <options> <arguments ...>

  create          host1:port1 ... hostN:portN
                  -<span class="ruby">-replicas <arg>
</span>  check           host:port
  info            host:port
  fix             host:port
                  -<span class="ruby">-timeout <arg>
</span>  reshard         host:port
                  -<span class="ruby">-from <arg>
</span>                  -<span class="ruby">-to <arg>
</span>                  -<span class="ruby">-slots <arg>
</span>                  -<span class="ruby">-yes
</span>                  -<span class="ruby">-timeout <arg>
</span>                  -<span class="ruby">-pipeline <arg>
</span>  rebalance       host:port
                  -<span class="ruby">-weight <arg>
</span>                  -<span class="ruby">-auto-weights
</span>                  -<span class="ruby">-use-empty-masters
</span>                  -<span class="ruby">-timeout <arg>
</span>                  -<span class="ruby">-simulate
</span>                  -<span class="ruby">-pipeline <arg>
</span>                  -<span class="ruby">-threshold <arg>
</span>  add-node        new_host:new_port existing_host:existing_port
                  -<span class="ruby">-slave
</span>                  -<span class="ruby">-master-id <arg>
</span>  del-node        host:port node_id
  set-timeout     host:port milliseconds
  call            host:port command arg arg .. arg
  import          host:port
                  -<span class="ruby">-from <arg>
</span>                  -<span class="ruby">-copy
</span>                  -<span class="ruby">-replace
</span>  help            (show this help)

For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster.
</code>

以下演示用redis-trib建立3个master，每一个带一个slave。然后执行resharding，再增加一个master和slave

<code class=" hljs sql"># 建立8个实例
$ for port in 5000 5001 5002 5003 5004 5005 5006 5007; <span class="hljs-operator"><span class="hljs-keyword">do</span> 
  redis-server --port ${port} --cluster-enabled yes --cluster-config-file nodes-${port}.conf --cluster-node-timeout <span class="hljs-number">2000</span> --cluster-slave-validity-factor <span class="hljs-number">10</span> --cluster-migration-barrier <span class="hljs-number">1</span> --cluster-require-<span class="hljs-keyword">full</span>-coverage yes --dbfilename dump-${port}.rdb  --daemonize yes
done

# 建立<span class="hljs-number">3</span>个master，每一个带一个slave
[redis@tt12c redis-<span class="hljs-number">3.0</span><span class="hljs-number">.7</span>]$ ./src/redis-trib.rb <span class="hljs-keyword">create</span> --replicas <span class="hljs-number">1</span> <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">5000</span> <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">5001</span> <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">5002</span> <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">5003</span> <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">5004</span> <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">5005</span>
>>> Creating cluster
>>> Performing hash slots allocation <span class="hljs-keyword">on</span> <span class="hljs-number">6</span> nodes...
<span class="hljs-keyword">Using</span> <span class="hljs-number">3</span> masters:
<span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">5000</span>
<span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">5001</span>
<span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">5002</span>
Adding replica <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">5003</span> <span class="hljs-keyword">to</span> <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">5000</span>
Adding replica <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">5004</span> <span class="hljs-keyword">to</span> <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">5001</span>
Adding replica <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">5005</span> <span class="hljs-keyword">to</span> <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">5002</span>
M: efcf9b94fb98895fb34c62b65d6641575c116da1 <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">5000</span>
   slots:<span class="hljs-number">0</span>-<span class="hljs-number">5460</span> (<span class="hljs-number">5461</span> slots) master
M: <span class="hljs-number">3</span>bfee1a4fc5ae221fc1d25b1daf0a057fc84edcf <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">5001</span>
   slots:<span class="hljs-number">5461</span>-<span class="hljs-number">10922</span> (<span class="hljs-number">5462</span> slots) master
M: <span class="hljs-number">17</span>aca50672f0f14280c4c507ab2af459caeb05d7 <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">5002</span>
   slots:<span class="hljs-number">10923</span>-<span class="hljs-number">16383</span> (<span class="hljs-number">5461</span> slots) master
S: fc9651cb5a5c0f567a5849d3eacae95c343ef569 <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">5003</span>
   replicates efcf9b94fb98895fb34c62b65d6641575c116da1
S: bd17ac5db6636f1326241e00706e7e2f8fd3da5c <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">5004</span>
   replicates <span class="hljs-number">3</span>bfee1a4fc5ae221fc1d25b1daf0a057fc84edcf
S: adeb49ad5120fea3d65f6f9de87d88805ab4c52e <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">5005</span>
   replicates <span class="hljs-number">17</span>aca50672f0f14280c4c507ab2af459caeb05d7
Can I <span class="hljs-keyword">set</span> the above configuration? (type <span class="hljs-string">'yes'</span> <span class="hljs-keyword">to</span> accept): yes
>>> Nodes configuration updated</span></code>

Kenyataan：

Kandungan artikel ini disumbangkan secara sukarela oleh netizen, dan hak cipta adalah milik pengarang asal. Laman web ini tidak memikul tanggungjawab undang-undang yang sepadan. Jika anda menemui sebarang kandungan yang disyaki plagiarisme atau pelanggaran, sila hubungi admin@php.cn

Artikel sebelumnya：Hbase的协处理器Artikel seterusnya：MySql5.7.12免安装版配置以及服务无法启动问题解决方法

Artikel berkaitan

Lihat lagi