redis data types and application scenarios
- angryTomOriginal
- 2020-02-13 13:23:368332browse
redis data types and application scenarios
Redis supports 5 data types: string (string), hash (hash), list (list), set (set) and zset (sorted set: ordered set).
1. String
Introduction: Strings data type is the most commonly used and simple key-value type, and ordinary key/value storage can be classified into this category. value can be not only a string, but also a number. Because it is binary safe, you can store the contents of an image file as a string. Redis's string can fully realize the current functions of memcached and is more efficient. In addition to providing the same get, set, incr, decr and other operations as Memcached, Redis also provides the following additional operations:
1. Get the length of the string
2. Append to the string Content
3. Set and get a certain content of the string
4. Set and get a certain bit of the string
5. Set a series in batches Contents of string
Common commands: set, get, decr, incr, mget, etc.
Application scenarios:
1. All scenarios where Memcached and CKV are applied. Strings and numbers are accessed directly. Structured data needs to be serialized first and then set to value; accordingly, deserialization is required after getting value.
2. You can use the INCR, INCRBY, DECR, DECRBY and other instructions of redis to achieve the effect of atomic counting. That is, it can be used to realize business statistical counting requirements. It can also be used to implement idmaker, which generates globally unique ids.
3. Store session key and implement a distributed session system. Redis key can easily set the expiration time, which is used to realize the automatic expiration of session key. When verifying the skey, first route to the corresponding redis based on the uid. If the skey cannot be obtained, it means that the skey has expired and you need to log in again; if the skey is obtained and the verification passes, the expiration time of the skey can be upgraded.
4. Set nx or SetNx, Set only when the key does not exist. It can be used to elect the Master or implement distributed locks: all Clients constantly try to use SetNx master myName to register for the Master, and the successful one constantly uses Expire to refresh its expiration time. If the Master dies, the key will become invalid, and a new round of snatching will occur on the remaining nodes.
5. With the Lua script supported by redis2.6, two more secure distributed locks can be implemented: one is suitable for scenarios where processes compete but a single process always acquires the lock and processes it. Unless the original processing process hangs up and the lock expires, the lock will be acquired by other processes. No need to actively unlock it. Implemented through Lua scripts of get, expire/pexpire, setnx ex| px; a scenario suitable for each process to compete to acquire locks and process them. Obtain the lock through set nx ex| px. After use, you need to judge by get first and then release the lock by del. Otherwise, the lock cannot be acquired before the lock expires.
6. GetSet, sets the new value and returns the old value. For example, to implement a counter, you can use GetSet to obtain the count and reset it to 0.
7. GetBit/SetBit/BitOp/BitCount, how to play BitMap, for example, when counting the number of unique visiting users today, each registered user has an offset. If he comes in today, set his bit to 1. Use BitCount to get the total number of people today.
8. Append/SetRange/GetRange/StrLen, extend, replace, intercept and find the length of text, which is very useful for specific data formats.
Implementation method: String is stored in redis as a string by default, which is referenced by redisObject. When encountering incr, decr and other operations, it will be converted into a numerical type for calculation. At this time, the encoding field of redisObject is int. .
2. Hash
Introduction: Hash stores the mapping between strings and string values. Hash stores various attributes of the object in the Map, and can only read/update certain attributes of the object. In this way, some properties that are too long can be left alone. In addition, different modules can only update the properties they care about without causing overwriting conflicts due to concurrency with each other.
Commonly used commands: hget, hset, hgetall, etc.
Application scenarios:
1. Store structured data, such as user information. In Memcached or CKV, for user information such as the user's nickname, age, gender, points, etc., we need to serialize it first and store it as a string value. At this time, when we need to modify one of the items, we usually need to After the value is taken out and deserialized, the value of an item is modified, and then serialized and stored back. This not only increases the overhead, but is also not suitable for situations where concurrent operations are possible (for example, two concurrent operations require modification of points). The Hash structure of Redis allows you to modify only a certain attribute value just like updating an attribute in the database. As shown below:
2. Key is the user ID, value is a Map, the key of this Map is the attribute name of the member, and value is the attribute value, so that the modification and access of the data can be directly done through its internal The Key of the Map (the key of the internal Map is called field in Redis), that is, the corresponding attribute data can be manipulated through the key (user ID) field (attribute label). There is no need to store data repeatedly, and it will not cause serialization. and the issue of concurrent modification control.
3. However, it should be noted here that Redis provides an interface (hgetall) to directly obtain all attribute data. However, if there are many members of the internal Map, it involves traversing the entire internal Map. Due to the Redis single-threaded model Therefore, this traversal operation may be time-consuming and may not respond to requests from other clients at all. This requires special attention.
4. Can be used to build indexes. For example, User objects, in addition to id, sometimes also need to be queried by name. You can build a Hash with the key user:name:id. When inserting the User object (set user:101{"id":101,"name":"calvin "}), by the way, insert an entry into this hash (hset user:name:id calvin 101). At this time, calvin is used as a key in the hash, with a value of 101. When querying by name, use hgetuser:name:id calvin to get the id from the key named calvin. If you need to use multiple indexes to find a certain piece of data, you can use one hash key to avoid using multiple string keys to store index values.
5. HINCRBY can also be used to implement idmaker. Compared with the string type idmaker, each type requires a key, and the hash type only needs one key.
Implementation method:
The Redis Hash corresponding to the Value is actually a HashMap. There will be two different implementations. When the Hash has fewer members, Redis will use a one-dimensional array similar to it to save memory. Method to compactly store without using the real HashMap structure. The encoding of the corresponding value redisObject is zipmap. When the number of members increases, it will automatically be converted into a real HashMap. At this time, the encoding is ht.
3. List
Introduction: List is a two-way linked list, supporting two-way Pop/Push. The rules of the world generally push from the left end, and Pop from the right end - LPush/RPop. , and there is also a Blocking version BLPop/BRPop, where the client can block until a message arrives. There is also RPopLPush/BRPopLPush, which pops up and returns to the client. At the same time, it pushes itself into another list, and LLen gets the length of the list. There are also operations by value: LRem (delete elements by value), LInsert (insert before and after elements of a certain value), the complexity is O(N), N is the length of the List, because the value of the List is not unique, so To traverse all elements, Set only takes O(log(N)).
Operations performed by subscript: The subscript starts from 0, the queue is counted from left to right, and when the subscript is a negative number, it is calculated from right to left. LSet, sets the element value by subscript. LIndex, returns the element by subscript. LRange, unlike POP that directly bounces away elements, only returns elements with a subscript in the list. It is a favorite for paging. LTrim, limits the size of the List, for example, only retains the latest 20 messages. The complexity is also O(N), where N of LSet is the length of the List, N of LIndex is the value of the subscript, and N of LRange is the value of start to list the number of elements. Because it is a linked list rather than an array, it is accessed by subscript In fact, you need to traverse the linked list, unless the subscripts happen to be the head and tail of the team. N of LTrim is the number of elements to remove.
Commonly used commands: lpush, rpush, lpop, rpop, lrange, etc.
Application scenarios:
1. Various lists, such as Twitter’s follow list, fan list, etc., the latest news rankings, comments on each article, etc. can also be implemented using Redis’ list structure .
2 In the message queue, you can use the PUSH operation of Lists to store tasks in Lists, and then the worker thread uses the POP operation to take out the tasks for execution. The message queue here does not have an ack mechanism. What if the consumer gives the task to Pop and crashes before finishing it? One solution is to add an additional sorted set. When distributing, send it to both the list and the sorted set at the same time. The distribution time is used as the score. After the user completes the task, he must use ZREM to eliminate the jobs in the sorted set and periodically remove the jobs from the sorted set. Remove the unfinished tasks that have timed out and put them back into the list. Another approach is to add an extra list for each worker, use RPopLPush when popping up tasks, put the job into the worker's own list at the same time, and use LREM to eliminate it when completed. If the cluster management (such as zookeeper) finds that the worker has died, the worker's list content will be returned to the main list.
3 Using LRANGE can easily realize the paging function of list content.
4. The operation of getting the latest N data: LPUSH is used to insert a content ID and store it as a keyword at the head of the list. LTRIM is used to limit the number of items in the list to a maximum of 5000. If the amount of data the user needs to retrieve exceeds this cache capacity, then the request needs to be sent to the database.
Implementation method:
The implementation of Redis list is a two-way linked list, which can support reverse search and traversal, which is more convenient to operate, but it brings some additional memory overhead. Redis internal Many implementations, including send buffer queues, etc., also use this data structure.
4. Set
Introduction: It is an unordered set. The elements in the set have no order and are not repeated. Putting duplicate elements into a Set will automatically remove them.
Common commands:
sadd, spop, smembers, sunion, etc.
Application scenarios:
1. Some lists that need to be deduplicated, and set provides an important interface for judging whether a member is in a set collection, which is also something that list cannot provide. .
2. Some collective data can be stored. For example, in a Weibo application, all followers of a user can be stored in a collection, and all fans can be stored in a collection. Redis also provides operations such as intersection, union, and difference for collections, which can be very convenient to implement functions such as joint attention, common preferences, and second-degree friends. For all the above collection operations, you can also use different command selections. Return the results to the client or save them in a new collection. For another example, QQ has a social function called "Friend Tags". You can tag your friends, such as "big beauty", "tycoon", "Ouba", etc. Here you can also store each user's tags in within a collection.
3. If you want to know how many specific registered users or IP addresses have visited a certain page, you can do this: SADD page:day1:
Implementation method:
The internal implementation of set is a HashMap whose value is always null. In fact, it is quickly sorted by calculating hash. This is also what set can provide to determine whether a member is Reason within the set.
5. Sorted Set
Introduction: Sorted set, compared with set, the score of the element is also provided when the element is put into the set, and it can be automatically sorted according to the score. .
Commonly used commands:
zadd, zrange, zrem, zcard, etc.
Usage scenarios:
1. Store an ordered and non-duplicate set Lists, such as Twitter's public timeline, can be stored with the publication time as the score, so that they are automatically sorted by time when retrieved.
2. You can make a weighted queue. For example, the score of ordinary messages is 1, and the score of important messages is 2. Then the worker thread can choose to obtain the work tasks in the reverse order of the score. Prioritize important tasks.
3. Ranking related: ZADD leaderboard
4. News is sorted according to user votes and time. Score during ZADD = points / time^alpha. In this way, user votes will dig out the news accordingly, but time will bury the news according to a certain index. .
5. Expired item processing: Use unix time as a key to keep the list sorted by time. Retrieve current_time and time_to_live to complete the difficult task of finding expired items. Another background task uses ZRANGE...WITHSCORES to query and delete expired entries.
Implementation method:
Redis sorted set internally uses HashMap and skip list (SkipList) to ensure the storage and ordering of data. HashMap places the mapping from members to scores, and skip All members are stored in the table, and the sorting basis is the score stored in the HashMap. Using the jump table structure can achieve higher search efficiency, and is relatively simple to implement.
For more redis knowledge, please pay attention to the redis database tutorial column.
The above is the detailed content of redis data types and application scenarios. For more information, please follow other related articles on the PHP Chinese website!