redis cluster construction and monitoring environment

Achieve a cluster of 3 masters and 3 slaves. Single machine IP: 192.168.40.128

Basic cluster construction

Simple download

Via wget http://download.redis.io/releases/redis-4.0.10.tar.gz
Unzip tar zxvf redis-4.0.10.tar.gz
Specify the installation path, switch to root user and execute make && make PREFIX=/usr/ local/redis install, there may be a problem of insufficient permissions, sudo will also report an error, use root directly to operate.

Install compilation tools

sudo apt-get update
sudo apt-get install gcc
sudo apt-get install make
sudo apt-get install tcl

#Create the redis cluster folder

Because it is /usr, So always operate under root permissions
##cd /usr/local/redis
mkdir cluster
cd cluster
mkdir 7000 7001 7002 7003 7004 7005

Modify the configuration file

Copy the config file in redis conf to six folders, and modify the following content

# 端口号  
port 7000  
# 后台启动  
daemonize yes  
# 开启集群  
cluster-enabled yes  
#集群节点配置文件  
cluster-config-file nodes-7000.conf  
# 集群连接超时时间  
cluster-node-timeout 5000  
# 进程pid的文件位置  
pidfile /home/ubuntu/redis-4.0.10/pid/redis-7000.pid
#工作文件夹
dir "/home/ubuntu/redis-4.0.10/working"
# 开启aof  
appendonly yes  
# aof文件路径  
appendfilename "appendonly-7005.aof"  
# rdb文件路径  
dbfilename dump-7000.rdb

The bind in the redis configuration file specifies the network card IP of the redis server, which is the IP of the redis server

Startup script

cd / home/ubuntu/redis-4.0.10/
touch start.link.shFor easy operation, create a script
Modify the startup script to

#!/bin/bash
export BASE_FLOD="/usr/local/redis"
{BASE_FLOD}/bin/redis-server /usr/local/redis/cluster/7000/redis.conf
/usr/local/redis/bin/redis-server /usr/local/redis/cluster/7001/redis.conf
/usr/local/redis/bin/redis-server /usr/local/redis/cluster/7002/redis.conf
/usr/local/redis/bin/redis-server /usr/local/redis/cluster/7003/redis.conf
/usr/local/redis/bin/redis-server /usr/local/redis/cluster/7004/redis.conf
/usr/local/redis/bin/redis-server /usr/local/redis/cluster/7005/redis.conf
#cd src
#./redis-trib.rb create --replicas 1 192.168.40.128:7000 192.168.40.128:7001 192.168.40.128:7002 192.168.40.128:7003 192.168.40.128:7004 192.168.40.128:7005

The comments are to simplify the initial startup. The ip needs to be bound to the bind attribute in redis.conf configured for each node. Consistent, after startup, you can use the ps -ef | grep redis command to query whether the corresponding thread is started

Cluster startup

The associated program is written in ruby. So to set up rudy's operating environment, you need to install rudbygem
sudo apt-get install ruby rubygems -y
gem install redis, running here will feel very slow, you need to wait patiently, in the redis installation directory, src folder redis-trib.rb
run
redis-trib. rb create --replicas 1 192.168.40.128:7000 192.168.40.128:7001 192.168.40.128:7002 192.168.40.128:7003 192.168.40.128:7004 192.168.40.12 8:7005, check whether the configured information is correct, there is no direct Just yes. [OK] All 16384 slots covered. means the group connection is started successfully.

Node check, restart

Check the cluster running status: use the command

./redis-trib.rb check 192.168.40.128:7000, check the status of the cluster

Performance test

Comes with the test tool redis-benchmark

redis-benchmark - h 192.168.40.128 -p 6379 -c 100 -n 100000100 concurrent connections, 100000 requests, detect the performance of the redis server whose host is localhost and the port is 6379.
redis-benchmark -h 192.168.40.128 -p 6379 -q -d 100Test the performance of accessing data packets with a size of 100 bytes.
redis-benchmark -t set,lpush -n 100000 -qOnly test the performance of certain operations.
redis-benchmark -n 100000 -q script load "redis.call('set','foo','bar')"Only test some Performance of numerical access.

Cluster password setting

No password is required for the initial cluster establishment. After the startup is completed, first check whether the configuration file of each node has read and write permissions. If There is no read and write permissions. You need to modify the read and write permissions by chmod. Connect each node separately for settings through

./redis-cli -c -p port
config set masterauth password
config set requirepass password
config rewrite

If you find that the connection cannot be reached after restarting, modify the startup script redis-.sh line 99 and configure Startup script password startup

@r = Redis.new(:host => @info[:host], :port => @info[:port], :timeout => 60,:password => "yangfan@1995")

Code test

/*
 *集群连接测试
 */

@Test
public void testJedisCluster() {
    Set<HostAndPort> nodes = new LinkedHashSet<>();
    //所有主机节点ip和端口
    nodes.add(new HostAndPort("192.168.40.128", 7000));
    nodes.add(new HostAndPort("192.168.40.128", 7001));
    nodes.add(new HostAndPort("192.168.40.128", 7002));
    nodes.add(new HostAndPort("192.168.40.128", 7003));
    nodes.add(new HostAndPort("192.168.40.128", 7004));
    nodes.add(new HostAndPort("192.168.40.128", 7005));
    //没有密码
    //JedisCluster cluster = new JedisCluster(nodes);
    //添加密码调用
    JedisCluster cluster = new JedisCluster(nodes, 5000, 5000, 10, "yangfan@1995", new GenericObjectPoolConfig());
    //cluster.zadd("test_1", String.valueOf(""),"id_2");
    System.out.println(cluster.zscore("test_1", "id_1"));
    try {
        cluster.close();
    } catch (IOException e) {
        e.printStackTrace();
    }
}

The relationship between master-slave mode, sentry, and cluster

The master-slave mode specifies the replication and persistence relationship, and the master-slave backup relationship.
Sentinel: When the master database encounters an exception and interrupts the service, developers can manually method to select a slave database to upgrade to the master database so that the system can continue to provide services. It is mainly a detection tool to solve the problem of manual switching of master-slave relationship during master-slave replication, and can automatically switch master-slave.
使用哨兵，redis每个实例也是全量存储，每个redis存储的内容都是完整的数据，浪费内存且有木桶效应。为了最大化利用内存，可以采用集群，就是分布式存储。即每台redis存储不同的内容，共有16384个slot。每个redis分得一些slot，hash_slot = crc16(key) mod 16384 找到对应slot，键是可用键，如果有{}则取{}内的作为可用键，否则整个键是可用键集群至少需要3主3从，且每个实例使用不同的配置文件，主从不用配置，集群会自己选。

监控部署

RedisLive搭建部署

运行环境部署

git clone https://github.com/kumarnitin/RedisLive.git
下载redislive,解压缩unzip -o -d /home/ubuntu/ RedisLive-master.zip

进入src文件夹，复制example文件，编辑

"RedisServers":
[
    {
    "server": "192.168.40.128",
    "port" : 7000,
    "password" : "yangfan@1995"
    },
    //...多个监听
],

"DataStoreType" : "redis",

"RedisStatsServer": //存储的redis监听接口
{
    "server" : "127.0.0.1",
    "port" : 6379
},

"SqliteStatsStore" :
{
    "path":  "/home/ubuntu/redis-4.0.10/working/redislive.db" //进行存储的文件
} }

ubuntu@ubuntu:~/redis-4.0.10$ mkdir pid
ubuntu@ubuntu:~/redis-4.0.10$ mkdir log
ubuntu@ubuntu:~/redis-4.0.10$ mkdir working
//保存aof，rdb，node-config文件。
RedisLive分为两部分，其中一部分为监控脚本，另一部分为web服务，所以需要分别启动。`./redis-monitor.py
--duration=120`./redis-live.py

Q&A

redis.clients.jedis.exceptions.JedisNoReachableClusterNodeException: No reachable node in cluster redis node的redis.conf 绑定ip设置为指定的redis节点ip，启动集群时只用指定ip启动，不使用192.168.40.128
connect refuse 关闭防火墙
No module named redis