Redis method of traversing all keys

Redis key command is used to manage redis keys. This article will introduce to you two methods in redis for traversing all keys in redis - KEYS pattern and SCAN cursor. I hope it will be of some help to you.

When we need to traverse all Redis keys or keys in a specified mode, the first thing that comes to mind is the KEYS command:

KEYS pattern

The official website has a tip for the KEYS command : KEYS is very fast. For example, it takes 40 milliseconds for Redis to execute a query in a database with 1 million keys. But using it in a large database may still cause performance problems. If you need to find specific KEYS from a data set, you'd better use Redis's collection structure SETS instead.

The KEYS command is very simple to use.

redis> MSET one 1 two 2 three 3 four 4
OK
redis> KEYS *o*
1) "four"
2) "one"
3) "two"
redis> KEYS t??
1) "two"
redis> KEYS *
1) "four"
2) "three"
3) "one"
4) "two"
redis>

But since the KEYS command returns all matching keys at once, when there are too many keys in redis, it is a hidden danger for memory consumption and the redis server.

For Redis 2.8 The above version provides us with a better command SCAN for traversing keys. The basic format of this command is:

SCAN cursor [MATCH pattern] [COUNT count]

SCAN. Each execution will only return a small number of elements, so it can be used in production environments without appearing. Issues like KEYS or SMEMBERS commands that may block the server.

The SCAN command is a cursor-based iterator. This means that every time the command is called, it needs to use the cursor returned by the previous call as the cursor parameter of the call, so as to continue the previous iteration process

When the cursor parameter of the SCAN command (i.e. cursor) When set to 0, the server will start a new iteration, and when the server returns a cursor with a value of 0 to the user, it means that the iteration has ended.

Simple iteration demonstration:

redis 127.0.0.1:6379> scan 0
1) "17"
2)  1) "key:12"
    2) "key:8"
    3) "key:4"
    4) "key:14"
    5) "key:16"
    6) "key:17"
    7) "key:15"
    8) "key:10"
    9) "key:3"
   10) "key:7"
   11) "key:1"
redis 127.0.0.1:6379> scan 17
1) "0"
2) 1) "key:5"
   2) "key:18"
   3) "key:0"
   4) "key:2"
   5) "key:19"
   6) "key:13"
   7) "key:6"
   8) "key:9"
   9) "key:11"

In the above example, the first iteration uses 0 as the cursor, indicating the start of a new iteration. The second iteration uses the cursor 17 returned in the first iteration as the new iteration parameter.

Obviously, the return value of the SCAN command is an array containing two elements. The first array element is the new cursor used for the next iteration, and the second array element is another array. This array contains all the elements being iterated.

Note: The returned cursor is not necessarily incrementing. The cursor returned may be smaller than the previous one.

When the SCAN command is called for the second time, the command returns cursor 0, which indicates that the iteration has ended and the entire data set has been completely traversed.

full iteration: Start a new iteration with 0 as the cursor, and keep calling the SCAN command until the command returns cursor 0. We call this process a complete traversal.

The SCAN incremental iteration command does not guarantee that each execution will return a given number of elements, and may even return zero elements, but as long as the cursor returned by the command is not 0, the application should not Treat the iteration as complete.

However, the number of elements returned by the command always complies with certain rules. For a large data set, the incremental iteration command may return up to dozens of elements each time; for a small enough data For a set, all keys may be returned in one iteration

COUNT option

For incremental iteration commands that do not guarantee the number of elements returned in each iteration, we can use the COUNT option, for The behavior of the command is adjusted to some extent. The purpose of the COUNT option is to let the user tell the iteration command how many elements should be returned from the data set in each iteration. Using the COUNT option is equivalent to a hint for incremental iteration commands. In most cases, this hint is more effective in controlling the number of return values.

Note: The COUNT option does not strictly control the number of keys returned, it can only be said to be a rough constraint. It is not necessary to use the same COUNT value in each iteration. The user can change the COUNT value in each iteration as needed. Just remember to use the cursor returned from the last iteration to the next iteration.

MATCH option

Similar to the KEYS command, the incremental iteration command is implemented by giving the MATCH parameter. By providing a glob-style mode parameter, the command only returns the given mode. matching elements.

MATCH option's pattern matching of elements is performed during the period after the command retrieves the elements from the data set and before returning the elements to the client, so if there are only a small number of elements and patterns in the iterated data set match, the iteration command may be executed multiple times without returning any elements.

The following is an example of this situation:

redis 127.0.0.1:6379> scan 0 MATCH *11*
1) "288"
2) 1) "key:911"
redis 127.0.0.1:6379> scan 288 MATCH *11*
1) "224"
2) (empty list or set)
redis 127.0.0.1:6379> scan 224 MATCH *11*
1) "80"
2) (empty list or set)
redis 127.0.0.1:6379> scan 80 MATCH *11*
1) "176"
2) (empty list or set)
redis 127.0.0.1:6379> scan 176 MATCH *11* COUNT 1000
1) "0"
2)  1) "key:611"
    2) "key:711"
    3) "key:118"
    4) "key:117"
    5) "key:311"
    6) "key:112"
    7) "key:111"
    8) "key:110"
    9) "key:113"
   10) "key:211"
   11) "key:411"
   12) "key:115"
   13) "key:116"
   14) "key:114"
   15) "key:119"
   16) "key:811"
   17) "key:511"
   18) "key:11"
redis 127.0.0.1:6379>

As can be seen, most of the above iterations do not return any elements. In the last iteration, we force the command to scan more elements for this iteration by setting the parameter of the COUNT option to 1000, so that the command returns more elements.

Based on the security of SCAN, it is recommended that everyone use the SCAN command instead of KEYS in the production environment. However, please note that this command was added after version 2.8.0. If your Redis is lower than this version , you need to upgrade Redis.

The following uses PHP code to demonstrate the use of the SCAN command:

<?php
 
$redis = new Redis();
 
$redis->connect(&#39;127.0.0.1&#39;, 6379);
 
 
/* 设置遍历的特性为不重复查找，该情况下扩展只会scan一次，所以可能会返回空集合 */
$redis->setOption(Redis::OPT_SCAN, Redis::SCAN_NORETRY);
 
$it = NULL;
$pattern = &#39;*&#39;;
$count = 50;  // 每次遍历50条，注意是遍历50条，遍历出来的50条key还要去匹配你的模式，所以并不等于就能够取出50条key
 
do
{
    $keysArr = $redis->scan($it, $pattern, $count);
 
    if ($keysArr)
    {
        foreach ($keysArr as $key)
        {
            echo $key . "\n";
        }
    }
 
} while ($it > 0);   //每次调用 Scan会自动改变 $it 值，当$it = 0时 这次遍历结束 退出循环
 
 
echo &#39;---------------------------------------------------------------------------------&#39; . "\n";
 
 
/* 设置扩展在一次scan没有查找出记录时 进行重复的scan 直到查询出结果或者遍历结束为止 */
$redis->setOption(Redis::OPT_SCAN, Redis::SCAN_RETRY);
 
$it = NULL;
$pattern = &#39;*&#39;;
$count = 50;  // 每次遍历50条，注意是遍历50条，遍历出来的50条key还要去匹配你的模式，所以并不等于就能够取出50条key
 
//这种用法下我们只需要简单判断返回结果是否为空即可, 如果为空说明遍历结束
while ($keysArr = $redis->scan($it, $pattern, $count))
{
    foreach ($keysArr as $key)
    {
        echo $key . "\n";
    }
}

Execution results:

[root@localhost php]# /usr/local/php/bin/php scan.php 
bm
bm2
h1
name
bit
bm1
places
cities
hhl
---------------------------------------------------------------------------------
bm
bm2
h1
name
bit
bm1
places
cities
hhl

注意：如果php执行报错 请升级到较新版本的Redis扩展。

更多redis知识请关注redis入门教程栏目。

The above is the detailed content of Redis method of traversing all keys. For more information, please follow other related articles on the PHP Chinese website!