How to use redis+python as a message queue

1. Use the List type of redis combined with lpush and brpop to achieve

Introduction

• First of all, the List of redis is equivalent to a queue, which can implement first-in-first-out Rule

• The reason brpop is used is because when there is nothing in the queue, it will block until there is a pop-up element in the queue or the wait times out

Simulation Problem:

• There are too many accesses, and the server processing speed is too slow. If the server feedbacks every time the user waits, the time is too long, the http connection times out, and a server error occurs.

Simulation implementation process:

• There is a client that constantly puts things (data) into the queue, using multi-threading to simulate a large number of users Access situation

• There is a server that continuously takes out prints from the queue and sets the sleep time for each print to 2 seconds

Redis’ List Structure introduction

key [value, value]
key 代表List的名字， [value, ...] 是值

Client client.py

import random
import threading
import redis
import config

lock = threading.Lock()
lock.acquire()
lock.release()


pool = redis.ConnectionPool(host=config.HOST, port=config.PORT, 
                            decode_responses=True, password=config.PASSWORD)

r = redis.Redis(connection_pool=pool)

# 客户往redis 中放数据
def fun1(redisObj):
    value = random.randint(0, 100)
    # 往ccc列表中存放
    print("开始发送数据:", value)
    redisObj.lpush("print",str(value))

for i in range(100):
    threading.Thread(target=fun1, args=(r,)).start()

Server server.py

import redis
import time
import config


pool = redis.ConnectionPool(host=config.HOST, port=config.PORT, decode_responses=True, password=config.PASSWORD)
r = redis.Redis(connection_pool=pool)
# 服务端不断的取
while True:
    value = r.brpop("print")
    time.sleep(2)
    print(value)

Problem review

We said before that there is a problem of blocking for too long and disconnecting Here are the

ways to solve the problem: treat the connection as a function, capture errors, and reconnect when a problem occurs.

import redis
import time
import config

def get_redis():
    pool = redis.ConnectionPool(host=config.HOST, port=config.PORT, decode_responses=True, password=config.PASSWORD)
    r = redis.Redis(connection_pool=pool)
    return r
# 服务端不断的取
r = get_redis()
    
while True:
    try:
        value = r.brpop("print")
        time.sleep(2)
        print(value)
    except Exception as e:
        print("等待超时重连")
        r = get_redis()

The above is the detailed content of How to use redis+python as a message queue. For more information, please follow other related articles on the PHP Chinese website!