search

Home  >  Q&A  >  body text

python redis multi-process use

class RedisClient(object):
    def __init__(self):
        pool = redis.ConnectionPool(host='127.0.0.1', port=6379)
        self.client = redis.StrictRedis(connection_pool=pool)

Write a redis client with a connection pool according to the documentation, and then generate an instance for global use.
It has been tested to be normal to share an instance in multiple threads.
But in the case of multiple processes, the test failed

class ProcessRdeisTest(Process):
    def __init__(self,client):
        self._client = client

Written like this, when executing start, an error will be reported and serialization cannot be performed.
changed to:

class ProcessRdeisTest(Process):
    def __init__(self):
        pass
    def run(self):
        self._client = RedisClient()
        while Ture:
            dosomething()
        

It works like this, but is this connection method correct? Is there a better way to achieve this?

In the main thread directly
process1 = ProcessRdeisTest('p1')
process1.start()
Called in this way

巴扎黑巴扎黑2730 days ago1119

reply all(1)I'll reply

  • typecho

    typecho2017-06-08 11:04:09

    Owner, python redis has its own connection pool:

    import redis
    import threading
    
    class RedisPool(object):
        __mutex = threading.Lock()
        __remote = {}
    
        def __new__(cls, host, passwd, port, db):
            with RedisPool.__mutex:
                redis_key = "%s:%s:%s" % (host, port, db)
                redis_obj = RedisPool.__remote.get(redis_key)
                if redis_obj is None:
                    redis_obj = RedisPool.__remote[redis_key] = RedisPool.new_redis_pool(host, passwd, port, db)
            return redis.Redis(connection_pool=redis_obj)
    
        def __init__(self, host, passwd, port, db):
            pass
    
        @staticmethod
        def new_redis_pool(host, passwd, port, db):
            redis_obj = redis.ConnectionPool(host=host, password=passwd,
                                             port=port, db=db, socket_timeout=3, max_connections=10) # max_connection default 2**31
            return redis_obj
    

    reply
    0
  • Cancelreply