Detailed explanation of redis persistent storage

redis database tutorialThe column introduces the persistent storage of redis.

Recommended: redis database tutorial

redis overview
REmote DIctionary Server (Redis) is a persistent database storage system based on key-value pairs. Redis is very similar to the famous Memcached cache service software, but redis supports more data storage types than memcached, including strings (strings), lists (lists), sets (collections) and sorted sets (ordered sets), etc.
These data types support push/pop, add/remove, intersection, union, difference and richer operations, and these operations are all atomic. On this basis, redis supports various different ways of sorting. Like the memcached cache service, in order to ensure efficiency, data is cached in memory to provide services. Different from memcached, the redis persistent cache service will also periodically write updated data to disk and append modified operation records to files for recording. What is more advantageous than memcached is that redis also supports master- Slave (master-slave) synchronization is very similar to the master-slave replication function of the relational database MySQL.
Redis is an open source log-type Key-Value database written in C language (more than 30,000 lines of code), supports the network, can be memory-based and persistent, and provides APIs in multiple languages. Since March 15, 2010, the development of Redis has been hosted by VMware.
The emergence of Redis software has made up for the shortcomings of key-value memory caching services such as memcached to a certain extent. In some cases, it can play a very good supplementary role to relational databases. redis provides Python, Ruby, Erlang, and PHP clients,

1.2 redis features

key-value键值类型存储
支持数据可靠存储及落地
单进程单线程高性能服务器
crash safe & recovery slow
单机qps可以达到10W
适合小数据量高速读写访问

1.3 Redis advantages

Unlike memcached, Redis can persist storage Data performance is very high: Redis can support read and write frequencies of more than 10W per second.
Rich data types: Redis supports binary Strings, Lists, Hashes, Sets and sorted Sets and other data type operations
Atomic: All operations of Redis are atomic, and Redis also supports all operations for several operations. Atomic execution after merging
Rich features: Redis also supports publish/subscribe (publish/subscribe), notifications, key expiration and other features. Redis supports cross-machine master-slave replication.

1.4 redis defects and pitfalls
There are glitches in system operation
The latency of different commands varies greatly
Large memory management overhead (set lower than 3/5 of physical memory)
Buffer io causes system OOM (memory overflow)

1.5 redis data type
As a key-value storage system database, Redis provides a key (Key) and value (value) mapping relationship. However, in addition to regular numerical values ​​or strings, Redis key values ​​can also be in one of the following forms. The following are the most commonly used data types:

String 字符串
Hash 哈希表
List 列表
Set 集合
Sorted set 有序集合

1.6 redis persistence

Typically, Redis stores data in memory or is configured to use virtual memory. Data persistence can be achieved in two ways: using snapshots to continuously write data in memory to disk, or using MySQL-like binlog logs (aof but not used for master-slave synchronization) to record each time Updated log. The former has higher performance, but may cause a certain degree of data loss; the latter has the opposite effect.

1.7 redis application scenario
The best application scenario of redis

• The best trial scenario of Redis is all data in-memory
• Redis update Multiple scenarios are used as a replacement for Memcached.
• Data is relatively important and businesses that have certain requirements for data consistency.
• When you need support for more data types besides key/value, it is more appropriate to use Redis.
• Need to provide master-slave synchronization and load balancing distributed application scenarios (redis master-slave synchronization)

1.8 Lessons from redis production

Must be Perform Master-slave master-slave synchronization configuration, which can be switched when a service failure occurs

Disable data persistence on the master, and only need to configure data persistence on the slave

The physical memory and virtual memory are insufficient. At this time, the dump keeps dying. After a long time, the machine hangs. This situation is a disaster!

When Redis physical memory usage exceeds 3/5 of the total memory capacity, it will become more dangerous, and swap will begin, resulting in large memory fragmentation

When the maximum memory is reached, keys with expiration time will be cleared, even if the key has not expired.

redis与DB同步写的问题，先写DB，后写redis，因为写内存基本上没有问题。

快速部署一个redis环境
2.1 Redis部署环境搭建

主机名	eth0	用途
Master-redis01	10.0.0.135	主Redis
Slave-redis02	10.0.0.136	从Redis

2.2 开始安装redis服务
在redis的官方网站（http://www.redis.io）下载最新的稳定版本redis。

wget -q http://download.redis.io/releases/redis-2.8.9.tar.gz

#在redis01和redis02都执行如下操作

[root@redis01 ~]# tar xf redis-2.8.9.tar -C /usr/src/
[root@redis01 ~]# cd /usr/src/redis-2.8.9/
[root@redis01 redis-2.8.9]# make MALLOC=jemalloc
[root@redis01 redis-2.8.9]# make PREFIX=/usr/local/redis install
[root@redis01 redis-2.8.9]# LANG=en
[root@redis01 redis-2.8.9]# tree /usr/local/redis/bin/
/usr/local/redis/bin/
├── redis-benchmark
├── redis-check-aof
├── redis-check-dump
├── redis-cli
└── redis-server

0 directories, 5 files

命令执行完成之后，会在/usr/local/redis/bin/目录下生成5个可执行文件，分别是：

redis-server,redis-cli,redis-benchmark,redis-check-aof,redis-check-dump

它们的作用如下：

redis-server    #Redis服务器的daemon启动程序
redis-cli       #Redis命令操作工具。当然，你也可以用telnet根据其纯文本协议来操作
redis-benchmark #Redis性能测试工具，测试Redis在你的系统及你的配置下的读写性能。
redis-check-aof #对更新日志appendonly.aof检查，是否可用，类似检查mysql binlog的工具
redis-check-dump    #用于本地数据库rdb文件的检查

2.3 配置并启动redis服务
（1）配置启动命令

操作过程：

[root@redis01 redis-2.8.9]# ln -s /usr/local/redis/bin/* /usr/local/bin/
（2）查看命令帮助：

[root@redis01 redis-2.8.9]# redis-server -h
Usage: ./redis-server [/path/to/redis.conf] [options]
       ./redis-server - (read config from stdin)
       ./redis-server -v or --version
       ./redis-server -h or --help
       ./redis-server --test-memory <megabytes>

Examples:
       ./redis-server (run the server with default conf)
       ./redis-server /etc/redis/6379.conf
       ./redis-server --port 7777
       ./redis-server --port 7777 --slaveof 127.0.0.1 8888
       ./redis-server /etc/myredis.conf --loglevel verbose

Sentinel mode:
       ./redis-server /etc/sentinel.conf --sentinel</megabytes>

（3）启动redis服务

操作过程：

#从源程序目录复制redis.conf到程序安装目录下

[root@redis01 redis-2.8.9]# cd /usr/src/redis-2.8.9/
[root@redis01 redis-2.8.9]# pwd
/usr/src/redis-2.8.9
[root@redis01 redis-2.8.9]# mkdir /usr/local/redis/conf
[root@redis01 redis-2.8.9]# cp redis.conf /usr/local/redis/conf/

#启动redis服务

[root@redis01 redis-2.8.9]# redis-server /usr/local/redis/conf/redis.conf &

#查看redis进程启动情况

[root@redis01 redis-2.8.9]# ps -ef | grep redis | grep -v grep
root       3169   1288  0 10:17 pts/0    00:00:00 redis-server *:6379

特别提示：

redis启动成功后，在最后会出现如下警示信息：

[3169] 02 Oct 10:17:30.689 # Server started, Redis version 2.8.9
[3169] 02 Oct 10:17:30.690 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
[3169] 02 Oct 10:17:30.690 * The server is now ready to accept connections on port 6379

#警示大概意思为：
overcommit_memory被设置为了0.如果内存不够的情况下后台保存可能会失败；要解决这个问题，需要在/etc/sysctl.conf配置文件中将vm.overcommit_memory设置为1;或者通过命令“sysctl vm.overcommit_memory=1”来修改。
因此，我们做一下处理后在启动redis进程

[root@redis01 redis-2.8.9]# pkill redis
[root@redis01 redis-2.8.9]# sysctl vm.overcommit_memory=1
vm.overcommit_memory = 1
[root@redis01 redis-2.8.9]# redis-server /usr/local/redis/conf/redis.conf &

经过处理后，再启动redis就没有任何警告了。
vm.overcommit_memory参数说明：
根据内核文档，该参数有三个值，分别是：
0:当用户空间请求更多的内存时，内核尝试估算出剩余可用的内存。
1:当设这个参数值为1时，内核允许超量使用内存直到用完为止，主要用于科学计算
2:当设这个参数值为2时，内核会使用一个绝不过量使用内存的算法，即系统整个内存地址空间不能超过swap+50%的RAM值，50%参数的设定是在overcommit_ratio中设定。

测试关闭redis服务的命令

redis-cli shutdown 关闭redis进程

[root@redis01 redis-2.8.9]# ps -ef | grep redis | grep -v grep
root       3200   1288  0 10:38 pts/0    00:00:08 redis-server *:6379                          
[root@redis01 redis-2.8.9]# redis-cli shutdown
[3200] 02 Oct 12:43:46.621 # User requested shutdown...
[3200] 02 Oct 12:43:46.621 * Saving the final RDB snapshot before exiting.
[3200] 02 Oct 12:43:46.630 * DB saved on disk
[3200] 02 Oct 12:43:46.631 # Redis is now ready to exit, bye bye...
[1]+  Done                    redis-server /usr/local/redis/conf/redis.conf
[root@redis01 redis-2.8.9]# ps -ef | grep redis | grep -v grep
[root@redis01 redis-2.8.9]# redis-server /usr/local/redis/conf/redis.conf

redis自启动脚本

[root@ser02 redis]# vim redserver.sh
#!/bin/bash
stop(){
/data/redis/bin/redis-cli -a redis shutdown
}
start(){
/data/redis/bin/redis-server /data/redis/conf/redis.conf &
}
conn(){
/data/redis/bin/redis-cli -a redis
}
case $1 in
   start)
   start
   ;;
   stop)
   stop
   ;;
   restart)
   stop
   start
   ;;
   conn)
   conn
   ;;
   *)
   echo "Usage:$0 (start|stop|restart)"
esac
[root@ser02 redis]# chmod +x redserver.sh 
[root@ser02 redis]# vim /etc/profile
export PATH=/data/redis/:$PATH
[root@ser02 redis]# source /etc/profile

redserver.sh  start ##开启redis服务

redserver.sh conn ##登录redis
2.4 通过客户端操作redis数据库
下面我们来简单操作一下数据库。
插入数据：设置一个key-value对

[root@redis01 redis-2.8.9]# redis-cli   #通过客户端连接本地redis 
127.0.0.1:6379> set id 001  #写入一条数据key（id），value（001）
OK
127.0.0.1:6379> get id  #取值key（id）
"001"   #显示key对应的值
127.0.0.1:6379> del id  #删除key（id）
(integer) 1     #1表示成功
127.0.0.1:6379> exists id   #验证key是否存在
(integer) 0     #0表示不存在
127.0.0.1:6379> get id  #取key的值
(nil)           #报错信息
127.0.0.1:6379> set user001 benet
OK
127.0.0.1:6379> set user002 yunjisuan
OK
127.0.0.1:6379> set user003 yun123
OK
127.0.0.1:6379> get user001
"benet"
127.0.0.1:6379> get user002
"yunjisuan"
127.0.0.1:6379> keys *  #查看redis里所有的key
1) "user003"
2) "user002"
3) "user001"

更多操作方式及命令帮助

（1）redis数据库的表模式

127.0.0.1:6379> keys *  #查看所有key
1) "user003"
2) "user002"
3) "user001"
127.0.0.1:6379> select 1    #切换到表1模式
OK
127.0.0.1:6379[1]> keys *   #查询所有key
(empty list or set)     #什么都没有
127.0.0.1:6379[1]> set name wangwu  #写入一个key-value对
OK
127.0.0.1:6379[1]> keys *   #查看所有key
1) "name"           #key（name）已经有了
127.0.0.1:6379[1]> get name #查看key（name）的值
"wangwu"
127.0.0.1:6379[1]> select 0 #切换回表0模式（初始模式）
OK
127.0.0.1:6379> keys *      #查看所有key
1) "user003"
2) "user002"
3) "user001"

（2）redis-cli客户端的远程连接及非交互式操作数据库

[root@redis01 redis-2.8.9]# redis-cli -h 10.0.0.135 -p 6379
10.0.0.135:6379> quit
[root@redis01 redis-2.8.9]# redis-cli -h 10.0.0.135 -p 6379 set aaa 111
OK
[root@redis01 redis-2.8.9]# redis-cli -h 10.0.0.135 -p 6379 get aaa
"111"

redis安全
（1）为redis客户端设置外部链接密码

警告：
因为redis速度相当快，所以在一台比较好的服务器下，一个外部的用户可以在1秒内进行上万次的密码尝试，这意味着你需要指定非常非常强大的密码来防止暴力破解。
设置密码：

[root@ser02 bin]# vim /data/redis/conf/redis.conf 
 requirepass redis

（2）将危险的命令改名
rename-command set “sset” #将set改名为sset

为php安装redis客户端扩展
（1）获取源码包

wget https://github.com/nicolasff/phpredis/archive/master.zip

（2）安装

[root@redis01 ~]# ls -l phpredis-master.tar.gz 
-rw-r--r--. 1 root root 164509 Oct  2 19:23 phpredis-master.tar.gz
[root@redis01 ~]# tar xf phpredis-master.tar.gz -C /usr/src/
[root@redis01 ~]# cd /usr/src/phpredis-master/
[root@redis01 phpredis-master]# /usr/local/php/bin/phpize
[root@redis01 phpredis-master]# ./configure --with-php-config=/usr/local/php/bin/php-config
[root@redis01 phpredis-master]# make && make install

vim /etc/php.ini
	extensions = /usr/lib64/php/modules/redis.so
	[root@ser02 modules]# systemctl restart httpd

测试：

[root@ser02 redis]# cd /var/www/html/
	[root@ser02 html]# vim 1.php 
	<?php   	$redis = new Redis();
  	$redis -> connect("192.168.25.151",4423);
  	$redis -> auth("redis");
  	$redis -> set("name","anliu");
  	$var = $redis -> get("name");
 	echo "$var";
	?>
	[root@ser02 html]# systemctl restart httpd

	
	[root@ser02 html]# php 1.php 
	anliu

安装Python redis客户端操作redis

wget https://pypi.python.org/packages/source/r/redis/redis-2.10.1.tar.gz
tar xf redis-2.10.1.tar.gz
cd redis-2.10.1
python setup.py install

开发python程序操作redis

在操作前请将之前redis配置文件里修改的redis命令注释掉，否则报错

[root@redis01 redis-2.10.1]# python
Python 2.6.6 (r266:84292, Nov 22 2013, 12:16:22) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import redis        #引用redis支持库
>>> r = redis.Redis(host='10.0.0.135',port='6379',password='yunjisuan') #建立redis数据库的连接对象（面向对象方式）
>>> r.set('name','benet')   #操作对象调用set方法写入数据
True
>>> r.get('name')           #操作对象调用get方式读取数据
'benet'
>>> r.dbsize()              #操作对象查看redis数据库的数据条数
1L
>>> r.keys()                #查看所有的key
['name']
>>> exit()                  #退出

2.11 通过Web界面连接Python程序展示redis
开发Python脚本

[root@redis01 scripts]# cat python-redis.py 
#/usr/bin/python

from wsgiref.simple_server import make_server
import redis

def get_redis():
    r = redis.Redis(host='10.0.0.135',port='6379',password='yunjisuan',db=0)
    r.set('name','yunyunyun')
    return r.get('name')
    
def hello_world_app(environ,start_response):
    status = '200 OK'   #HTTP Status
    headers = [('Content-type','text/plain')]   #HTTP Headers
    start_response(status,headers)
    
    # The returned object is going to be printed
    return get_redis()

httpd = make_server('',8000,hello_world_app)
print "Serving on port 8000..."

# Server until process is killed
httpd.serve_forever()

启动python脚本

注意关闭iptables

[root@redis01 scripts]# python python-redis.py 
Serving on port 8000...     #监听8000端口

The above is the detailed content of Detailed explanation of redis persistent storage. For more information, please follow other related articles on the PHP Chinese website!