Detailed introduction to redis application scenarios

Redis has created a new data storage idea. Using Redis, we don’t have to focus on how to put the elephant in the refrigerator when facing a database with monotonous functions. Instead, we use Redis’s flexible data structures and data operations to build different refrigerators for different elephants. Hope you like this metaphor.

1. Commonly used data types in Redis (recommended: redis video tutorial)

The most commonly used data types in Redis mainly include the following five types:

String , Hash, List, Set, Sorted set

Before describing these data types in detail, let’s first understand how these different data types are described in Redis’ internal memory management through a picture:

First of all, Redis uses a redisObject object internally to represent all keys and values. The main information of redisObject is as shown in the figure above: type represents the specific data type of a value object, encoding It is the way different data types are stored inside redis.

For example: type=string means that the value is stored as an ordinary string, then the corresponding encoding can be raw or int. If it is int, it means that the actual redis internally stores and represents this character according to the numerical type class. String, of course, the premise is that the string itself can be represented by a numerical value, such as: "123" "456" such strings.

A special explanation of the vm field is required here. Only when the virtual memory function of Redis is turned on can this field actually allocate memory. This function is turned off by default.

Through the above figure, we can find that Redis uses redisObject to represent all key/value data, which is a waste of memory. Of course, these memory management costs are mainly to provide a unified management interface for different data types of Redis. , the actual author also provides a variety of methods to help us save memory usage as much as possible, which we will discuss in detail later.

2. Application and implementation of various data types

Let’s first analyze the use and internal implementation of these five data types one by one:

1. String

String data structure is a simple key-value type. In fact, value is not only String, but also a number.

Commonly used commands: get, set, incr, decr, mget, etc.

Application scenarios: String is the most commonly used data type, and ordinary key/value storage can be classified into this category, which means that the current functions of Memcached can be fully realized and more efficient. You can also enjoy Redis's scheduled persistence, operation logs, and Replication functions. In addition to providing the same get, set, incr, decr and other operations as Memcached, Redis also provides the following operations:

• Get the string length

• Append content to the string

• Set and get a certain content of the string

• Set and get a certain bit of the string (bit)

• Set the contents of a series of strings in batches

Usage scenario: regular key-value caching application. Regular counts: number of Weibo posts, number of fans.

Implementation method: String is stored in redis as a string by default, which is referenced by redisObject. When encountering incr, decr and other operations, it will be converted into a numerical type for calculation. At this time, the encoding field of redisObject is int. .

2. Hash

Commonly used commands: hget, hset, hgetall, etc.

Application scenario:

Let’s give a simple example to describe the application scenario of Hash. For example, we want to store a user information object data, including the following information:

User ID For the search key, the stored value user object contains information such as name, age, birthday, etc. If it is stored using an ordinary key/value structure, there are two main storage methods:

The first method uses the user ID as the search key, encapsulates other information into an object and stores it in a serialized manner. The disadvantage of this method is that it increases the overhead of serialization/deserialization and requires modification. When one of the pieces of information is retrieved, the entire object needs to be retrieved, and the modification operation needs to protect concurrency, introducing complex issues such as CAS.

The second method is to store as many key-value pairs as there are members in the user information object, and use the name of the attribute corresponding to the user ID to obtain it as a unique identifier. Although serialization overhead and concurrency issues are eliminated for the value of the corresponding attribute, the user ID is stored repeatedly. If there is a large amount of such data, the memory waste is still very considerable.

Then the Hash provided by Redis solves this problem very well. The Hash of Redis actually stores the value internally as a HashMap, and provides an interface for direct access to the members of this Map, as shown below:

In other words, the key is still the user ID, and the value is a Map. The key of this Map is the attribute name of the member, and the value is the attribute value. In this way, the modification and access of the data can be directly done through the Key of the internal Map. (In Redis, the key of the internal Map is called field), that is, the corresponding attribute data can be manipulated through the key (user ID) field (attribute label). There is no need to store data repeatedly, and it will not cause serialization and concurrent modification. Control issues. Solved the problem very well.

It should be noted here that Redis provides an interface (hgetall) to directly obtain all attribute data. However, if there are many members of the internal Map, it involves traversing the entire internal Map. Due to the Redis single-threaded model Because of this, this traversal operation may be time-consuming, and other client requests will not respond at all. This requires special attention.

Usage scenario: Store some change data, such as user information, etc.

Implementation method:

As mentioned above, the Redis Hash corresponding to the Value is actually a HashMap. In fact, there are two different implementations. When the Hash has fewer members, Redis will use it to save memory. It is stored compactly in a way similar to a one-dimensional array without using the real HashMap structure. The encoding of the corresponding value redisObject is zipmap. When the number of members increases, it will automatically be converted into a real HashMap. At this time, the encoding is ht.

3. List

Commonly used commands: lpush, rpush, lpop, rpop, lrange, etc.

Application scenarios:

There are many application scenarios for Redis list, and it is also one of the most important data structures of Redis. For example, twitter's follow list, fan list, etc. can all be created using the Redis list structure. accomplish.

List is a linked list. I believe anyone with some knowledge of data structures should be able to understand its structure. Using the List structure, we can easily implement functions such as ranking of the latest news. Another application of List is the message queue.
You can use the PUSH operation of List to store tasks in the List, and then the worker thread uses the POP operation to take out the task for execution. Redis also provides an API for operating a certain segment in the List. You can directly query and delete elements of a certain segment in the List.

Implementation method:

The implementation of Redis list is a two-way linked list, which can support reverse search and traversal, which is more convenient to operate, but it brings some additional memory overhead. Redis internal Many implementations, including send buffer queues, etc., also use this data structure.

Redis's list is a doubly linked list in which each sub-element is of type String. You can add or delete elements from the head or tail of the list through push and pop operations, so that the List can be used as a stack or a queue. .

Usage scenarios:

Message queue system

Using list can build a queue system, using sorted set can even build a prioritized queue system.

For example: Using Redis as a log collector

It is actually a queue. Multiple endpoints write log information to Redis, and then a worker writes all logs to disk uniformly.

Operation to get the latest N data

Record the list of the first N latest logged-in user IDs, and the beyond range can be obtained from the database.

//把当前登录人添加到链表里
ret = r.lpush("login:last_login_times", uid)

//保持链表只有N位
ret = redis.ltrim("login:last_login_times", 0, N-1)

//获得前N个最新登陆的用户Id列表
last_login_list = r.lrange("login:last_login_times", 0, N-1)

For example, sina Weibo:

In Redis, our latest Weibo ID uses a resident cache, which is always updated. But we have limited it to no more than 5000 IDs, so our get ID function will always ask Redis. Only when the start/count parameters exceed this range, you need to access the database.

Our system does not "refresh" the cache like the traditional way, and the information in the Redis instance is always consistent. The SQL database (or other type of database on the hard disk) will only be triggered when the user needs to obtain "far" data, and the homepage or first comment page will not bother the database on the hard disk.

4. Set

Common commands:

sadd, spop, smembers, sunion, etc.

Application scenarios:

The external functions provided by Redis set are similar to list, which is a list function. The special thing is that set can automatically deduplicate. When you need to store a list data, When you don't want duplicate data, set is a good choice, and set provides an important interface for determining whether a member is in a set collection, which list cannot provide.

Set is a set. The concept of a set is a combination of a bunch of unique values. Some collective data can be stored using the Set data structure provided by Redis.

Case:

In the Weibo application, all followers of a user can be stored in a collection, and all his fans can be stored in a collection. Redis also provides operations such as intersection, union, and difference for collections, which can be very convenient to implement functions such as joint attention, common preferences, and second-degree friends. For all the above collection operations, you can also use different command selections. Return the results to the client or save them in a new collection.

Set is a set, an unordered set of String type. Set is implemented through hashtable. The concept is basically similar to the set in mathematics. It can be intersection, union, difference set, etc. The elements in set There is no order.

Method to realize:

set 的内部实现是一个 value永远为null的HashMap，实际就是通过计算hash的方式来快速排重的，这也是set能提供判断一个成员是否在集合内的原因。

使用场景：

交集，并集，差集：(Set)

//book表存储book名称

set book:1:name    ”The Ruby Programming Language”

set book:2:name     ”Ruby on rail”

set book:3:name     ”Programming Erlang”

//tag表使用集合来存储数据，因为集合擅长求交集、并集

sadd tag:ruby 1

sadd tag:ruby 2

sadd tag:web 2

sadd tag:erlang 3

//即属于ruby又属于web的书？

 inter_list = redis.sinter("tag.web", "tag:ruby") 

//即属于ruby，但不属于web的书？

 inter_list = redis.sdiff("tag.ruby", "tag:web") 

//属于ruby和属于web的书的合集？

 inter_list = redis.sunion("tag.ruby", "tag:web")

获取某段时间所有数据去重值

这个使用Redis的set数据结构最合适了，只需要不断地将数据往set中扔就行了，set意为集合，所以会自动排重。

5、Sorted Set

常用命令：

zadd,zrange,zrem,zcard等

使用场景：

Redis sorted set的使用场景与set类似，区别是set不是自动有序的，而sorted set可以通过用户额外提供一个优先级(score)的参数来为成员排序，并且是插入有序的，即自动排序。当你需要一个有序的并且不重复的集合列表，那么可以选择sorted set数据结构，比如twitter 的public timeline可以以发表时间作为score来存储，这样获取时就是自动按时间排好序的。

和Set相比，Sorted Set增加了一个权重参数score，使得集合中的元素能够按score进行有序排列，比如一个存储全班同学成绩的Sorted Set，其集合value可以是同学的学号，而score就可以是其考试得分，这样在数据插入集合的时候，就已经进行了天然的排序。

另外还可以用Sorted Set来做带权重的队列，比如普通消息的score为1，重要消息的score为2，然后工作线程可以选择按score的倒序来获取工作任务。让重要的任务优先执行。

实现方式：

Redis sorted set的内部使用HashMap和跳跃表(SkipList)来保证数据的存储和有序，HashMap里放的是成员到score的映射，而跳跃表里存放的是所有的成员，排序依据是HashMap里存的score,使用跳跃表的结构可以获得比较高的查找效率，并且在实现上比较简单。

三、Redis实际应用场景

1、显示最新的项目列表

下面这个语句常用来显示最新项目，随着数据多了，查询毫无疑问会越来越慢。

SELECT * FROM foo WHERE ... ORDER BY time DESC LIMIT 10

在Web应用中，“列出最新的回复”之类的查询非常普遍，这通常会带来可扩展性问题。这令人沮丧，因为项目本来就是按这个顺序被创建的，但要输出这个顺序却不得不进行排序操作。类似的问题就可以用Redis来解决。比如说，我们的一个Web应用想要列出用户贴出的最新20条评论。

在最新的评论边上我们有一个“显示全部”的链接，点击后就可以获得更多的评论。我们假设数据库中的每条评论都有一个唯一的递增的ID字段。我们可以使用分页来制作主页和评论页，使用Redis的模板，每次新评论发表时，我们会将它的ID添加到一个Redis列表：

LPUSH latest.comments <id></id>

我们将列表裁剪为指定长度，因此Redis只需要保存最新的5000条评论：

LTRIM latest.comments 0 5000

每次我们需要获取最新评论的项目范围时，我们调用一个函数来完成（使用伪代码）：

FUNCTION get_latest_comments(start, num_items):  
    id_list = redis.lrange("latest.comments",start,start+num_items - 1)  
    IF id_list.length <p>这里我们做的很简单。在Redis中我们的最新ID使用了常驻缓存，这是一直更新的。但是我们做了限制不能超过5000个ID，因此我们的获取ID函数会一直询问Redis。只有在start/count参数超出了这个范围的时候，才需要去访问数据库。</p><p>我们的系统不会像传统方式那样“刷新”缓存，Redis实例中的信息永远是一致的。SQL数据库（或是硬盘上的其他类型数据库）只是在用户需要获取“很远”的数据时才会被触发，而主页或第一个评论页是不会麻烦到硬盘上的数据库了。</p><p>2、排行榜应用，取TOP N操作</p><p>这个需求与上面需求的不同之处在于，取最新N个数据的操作以时间为权重，这个是以某个条件为权重，比如按顶的次数排序，这时候就需要我们的sorted set出马了，将你要排序的值设置成sorted set的score，将具体的数据设置成相应的value，每次只需要执行一条ZADD命令即可。</p><p>热门，排行榜应用：</p><pre class="brush:php;toolbar:false">//将登录次数和用户统一存储在一个sorted set里
zadd login:login_times 5 1
zadd login:login_times 1 2
zadd login:login_times 2 3
//当用户登录时，对该用户的登录次数自增1
ret = r.zincrby("login:login_times", 1, uid)
//那么如何获得登录次数最多的用户呢，逆序排列取得排名前N的用户
ret = r.zrevrange("login:login_times", 0, N-1)

另一个很普遍的需求是各种数据库的数据并非存储在内存中，因此在按得分排序以及实时更新这些几乎每秒钟都需要更新的功能上数据库的性能不够理想。典型的比如那些在线游戏的排行榜，比如一个Facebook的游戏，根据得分你通常想要：

- 列出前100名高分选手

- 列出某用户当前的全球排名

这些操作对于Redis来说小菜一碟，即使你有几百万个用户，每分钟都会有几百万个新的得分。模式是这样的，每次获得新得分时，我们用这样的代码：

ZADD leaderboard    

你可能用userID来取代username，这取决于你是怎么设计的。得到前100名高分用户很简单：

ZREVRANGE leaderboard 0 99

 用户的全球排名也相似，只需要：

ZRANK leaderboard

3、删除与过滤

我们可以使用LREM来删除评论。如果删除操作非常少，另一个选择是直接跳过评论条目的入口，报告说该评论已经不存在。 有些时候你想要给不同的列表附加上不同的过滤器。如果过滤器的数量受到限制，你可以简单的为每个不同的过滤器使用不同的Redis列表。毕竟每个列表只有5000条项目，但Redis却能够使用非常少的内存来处理几百万条项目。

4、按照用户投票和时间排序

排行榜的一种常见变体模式就像Reddit或Hacker News用的那样，新闻按照类似下面的公式根据得分来排序：score = points / time^alpha 因此用户的投票会相应的把新闻挖出来，但时间会按照一定的指数将新闻埋下去。下面是我们的模式，当然算法由你决定。

模式是这样的，开始时先观察那些可能是最新的项目，例如首页上的1000条新闻都是候选者，因此我们先忽视掉其他的，这实现起来很简单。每次新的新闻贴上来后，我们将ID添加到列表中，使用LPUSH + LTRIM，确保只取出最新的1000条项目。

有一项后台任务获取这个列表，并且持续的计算这1000条新闻中每条新闻的最终得分。计算结果由ZADD命令按照新的顺序填充生成列表，老新闻则被清除。这里的关键思路是排序工作是由后台任务来完成的。

5、处理过期项目

另一种常用的项目排序是按照时间排序。我们使用unix时间作为得分即可。 模式如下：

- 每次有新项目添加到我们的非Redis数据库时，我们把它加入到排序集合中。这时我们用的是时间属性，current_time和time_to_live。

- 另一项后台任务使用ZRANGE…SCORES查询排序集合，取出最新的10个项目。如果发现unix时间已经过期，则在数据库中删除条目。

6、计数

Redis是一个很好的计数器，这要感谢INCRBY和其他相似命令。我相信你曾许多次想要给数据库加上新的计数器，用来获取统计或显示新信息，但是最后却由于写入敏感而不得不放弃它们。

好了，现在使用Redis就不需要再担心了。有了原子递增（atomic increment），你可以放心的加上各种计数，用GETSET重置，或者是让它们过期。例如这样操作：

INCR user:<id> EXPIRE</id>

你可以计算出最近用户在页面间停顿不超过60秒的页面浏览量，当计数达到比如20时，就可以显示出某些条幅提示，或是其它你想显示的东西。

7、特定时间内的特定项目

另一项对于其他数据库很难，但Redis做起来却轻而易举的事就是统计在某段特点时间里有多少特定用户访问了某个特定资源。比如我想要知道某些特定的注册用户或IP地址，他们到底有多少访问了某篇文章。每次我获得一次新的页面浏览时我只需要这样做：

SADD page:day1:<page_id> <user_id></user_id></page_id>

当然你可能想用unix时间替换day1，比如time()-(time()%3600*24)等等。 想知道特定用户的数量吗？只需要使用

SCARD page:day1:<page_id></page_id>

需要测试某个特定用户是否访问了这个页面？

SISMEMBER page:day1:<page_id></page_id>

8、查找某个值所在的区间(区间无重合) ：(Sorted Set)

例如有下面两个范围，10－20和30－40

A_start 10, A_end 20

B_start 30, B_end 40

我们将这两个范围的起始位置存在Redis的Sorted Sets数据结构中，基本范围起始值作为score，范围名加start和end为其value值：

redis 127.0.0.1:6379> zadd ranges 10 A_start
(integer) 1
redis 127.0.0.1:6379> zadd ranges 20 A_end
(integer) 1
redis 127.0.0.1:6379> zadd ranges 30 B_start
(integer) 1
redis 127.0.0.1:6379> zadd ranges 40 B_end
(integer) 1

这样数据在插入Sorted Sets后，相当于是将这些起始位置按顺序排列好了。现在我需要查找15这个值在哪一个范围中，只需要进行如下的zrangbyscore查找：

redis 127.0.0.1:6379> zrangebyscore ranges (15 +inf LIMIT 0 1
1) "A_end"

这个命令的意思是在Sorted Sets中查找大于15的第一个值。（+inf在Redis中表示正无穷大，15前面的括号表示>15而非>=15）查找的结果是A_end，由于所有值是按顺序排列的，所以可以判定15是在A_start到A_end区间上，也就是说15是在A这个范围里。至此大功告成。

9、交集，并集，差集：(Set)

//book表存储book名称
set book:1:name    ”The Ruby Programming Language”
set book:2:name     ”Ruby on rail”
set book:3:name     ”Programming Erlang”

//tag表使用集合来存储数据，因为集合擅长求交集、并集
sadd tag:ruby 1
sadd tag:ruby 2
sadd tag:web 2
sadd tag:erlang 3

//即属于ruby又属于web的书？
 inter_list = redis.sinter("tag.web", "tag:ruby") 
//即属于ruby，但不属于web的书？
 inter_list = redis.sdiff("tag.ruby", "tag:web") 
//属于ruby和属于web的书的合集？
 inter_list = redis.sunion("tag.ruby", "tag:web")

更多redis相关文章请关注redis数据库教程栏目。

The above is the detailed content of Detailed introduction to redis application scenarios. For more information, please follow other related articles on the PHP Chinese website!