Detailed explanation of python coroutines (with examples)

This article brings you a detailed explanation of python coroutines (with examples). It has certain reference value. Friends in need can refer to it. I hope it will be helpful to you.

The switching of processes and threads takes time, and the current status of the thread process is saved to continue execution next time. In programs that do not require much CPU, that is, compared to IO-intensive programs, coroutines consume less resources than thread processes, switch faster, and are more suitable for IO-intensive programs. The coroutine is also single-threaded and cannot take advantage of the multi-core CPU. If you want to take advantage of the multi-core CPU, you can use the process-coroutine method, or the process-thread-coroutine method.

1. Simple implementation of coroutine

The principle of coroutine is to implement it through the generator, as follows: the program executes to line 19, executes the consumer function to line 13, next generator, executes producer The function stops at line 8, returns to the consumer function line 13 and continues execution. The loop reaches line 13 again, and the generator function will execute from where the last yield stopped. When the loop is completed in this way, the concurrency effect is completed.

But some people may say that if I use a loop to execute two functions in sequence, the result will be the same, as shown in the second example below. What I want to say here is that this does not preserve the execution position of the function. It simply switches to another function after the execution of one function. It cannot switch when encountering operations that require the CPU to wait. When encountering an operation that requires the CPU to wait, proactively give up the CPU, remember the execution position of the function, and switch back to continue execution next time, which can be regarded as concurrent operation and improve the concurrency effect of the program.

Coroutine simply implements producers and consumers

import time
def producer():
    while True:
        time.sleep(1)
        print("+++++ 1个包子", time.strftime("%X"))
        yield
def consumer():
    while True:
        next(prd)
        print("----- 1个包子", time.strftime("%X"))
if __name__ == "__main__":
    prd = producer()
    consumer()
# 输出结果
+++++ 1个包子 16:22:30
----- 1个包子 16:22:30
+++++ 1个包子 16:22:31
----- 1个包子 16:22:31
+++++ 1个包子 16:22:32
----- 1个包子 16:22:32
+++++ 1个包子 16:22:33
----- 1个包子 16:22:33

Sequential execution effect

import time
def producer():
    time.sleep(1)
    print("+++++ 1个包子", time.strftime("%X"))
def consumer():
    print("----- 1个包子", time.strftime("%X"))
if __name__ == "__main__":
    while True:
        producer()
        consumer()
# 输出结果
+++++ 1个包子 16:22:30
----- 1个包子 16:22:30
+++++ 1个包子 16:22:31
----- 1个包子 16:22:31
+++++ 1个包子 16:22:32
----- 1个包子 16:22:32
+++++ 1个包子 16:22:33
----- 1个包子 16:22:33

2, greenlet

The greenlet module needs to be installed, pip install greenlet. The greenlet principle is the encapsulation of generators. The greenlet class provides a method, switch: switch to the specified coroutine when switching is required.

from greenlet import greenlet
import time
def producer():
    while True:
        time.sleep(1)
        print("+++++ 1个包子", time.strftime("%X"))
        gr2.switch()  # 切换到gr2运行
def consumer():
    while True:
        print("----- 1个包子", time.strftime("%X"))
        gr1.switch()  # 切换到gr1运行
if __name__ == "__main__":
    gr1 = greenlet(producer)
    gr2 = greenlet(consumer)
    gr1.switch()  # 切换到gr1运行
# 输出结果
+++++ 1个包子 09:39:45
----- 1个包子 09:39:45
+++++ 1个包子 09:39:46
----- 1个包子 09:39:46
+++++ 1个包子 09:39:47
----- 1个包子 09:39:47

3. gevent

The gevent module also needs to be installed, pip install gevent. gevent is a re-encapsulation of gevent, which can automatically identify time-consuming operations and switch to other coroutines. Note that gevent will only switch coroutine running when it encounters a time-consuming operation. It will not actively switch if it does not encounter a time-consuming operation.

gevent.spawn(*args, **kwargs) The first parameter in the variable length parameter is the method fn executed by the coroutine, and the rest are the parameters of fn in turn. After starting the coroutine, call the join method.

There are two ways to identify time-consuming operations in the gevent module, ① Use overridden classes in the gevent module. For example, gevent.socket gevent.sleep ② Patching method, before all code. from gevent import monkey imports this module, and monkey.patch_all() calls this method.

It is recommended to use the second method, so that there is no need to change the code that has been written

Under normal circumstances, gevent will not recognize time-consuming operations

import time
import gevent
def producer():
    for i in range(3):
        time.sleep(1)
        print("+++++ 1个包子", name, time.strftime("%X"))
def consumer():
    for i in range(3):
        time.sleep(1)
        print("----- 1个包子", name, time.strftime("%X"))
if __name__ == "__main__":
    g1 = gevent.spawn(producer, "zhangsan")
    g2 = gevent.spawn(consumer, "lisi")
    g1.join()
    g2.join()
# 输出结果
+++++ 1个包子 zhangsan 10:42:38
+++++ 1个包子 zhangsan 10:42:39
+++++ 1个包子 zhangsan 10:42:40
----- 1个包子 lisi 10:42:41
----- 1个包子 lisi 10:42:42
----- 1个包子 lisi 10:42:43

gevent identifies time-consuming operation method 1, use the module in gevent

import time
import gevent
def producer():
    for i in range(3):
        gevent.sleep(1)
        print("+++++ 1个包子", time.strftime("%X"))
def consumer():
    for i in range(3):
        gevent.sleep(1)
        print("----- 1个包子", time.strftime("%X"))
if __name__ == "__main__":
    g1 = gevent.spawn(producer)
    g2 = gevent.spawn(consumer)
    g1.join()
    g2.join()
# 输出结果
+++++ 1个包子 10:43:04
----- 1个包子 10:43:04
+++++ 1个包子 10:43:05
----- 1个包子 10:43:05
+++++ 1个包子 10:43:06
----- 1个包子 10:43:06

gevent identifies time-consuming operation method 2, patch

import time
import gevent
from gevent import monkey
monkey.patch_all()
def producer():
    for i in range(3):
        time.sleep(1)
        print("+++++ 1个包子", time.strftime("%X"))
def consumer():
    for i in range(3):
        time.sleep(1)
        print("----- 1个包子", time.strftime("%X"))
if __name__ == "__main__":
    g1 = gevent.spawn(producer)
    g2 = gevent.spawn(consumer)
    g1.join()
    g2.join()
# 输出结果
+++++ 1个包子 10:44:04
----- 1个包子 10:44:04
+++++ 1个包子 10:44:05
----- 1个包子 10:44:05
+++++ 1个包子 10:44:06
----- 1个包子 10:44:06

When there are many open coroutines , it is a bit troublesome to call the join method one by one, so gevent provides a method joinall(), which can join all coroutines at once. The joinall() method passes a list containing all coroutines.

joinall

import time
import gevent
from gevent import monkey
monkey.patch_all()
def producer(name):
    for i in range(3):
        time.sleep(1)
        print("+++++ 1个包子", name, time.strftime("%X"))
def consumer(name):
    for i in range(3):
        time.sleep(1)
        print("----- 1个包子",  name, time.strftime("%X"))
if __name__ == "__main__":
    gevent.joinall([gevent.spawn(producer, "zhangsan"), gevent.spawn(consumer, "lisi")])
# 输出结果
+++++ 1个包子 zhangsan 10:51:34
----- 1个包子 lisi 10:51:34
+++++ 1个包子 zhangsan 10:51:35
----- 1个包子 lisi 10:51:35
+++++ 1个包子 zhangsan 10:51:36
----- 1个包子 lisi 10:51:36

4. Coroutine application, concurrent server

The server receives the client message and sends it back as it is

server

import socket
import gevent
from gevent import monkey
monkey.patch_all()
def fn(conn):
    msg = conn.recv(1024).decode("utf-8")
    print("服务的收到>>>", msg)
    conn.send(msg.encode("utf-8"))
sk = socket.socket()
sk.bind(("127.0.0.1", 8899))
sk.listen()
while True:
    conn, addr = sk.accept()
    print("已连接服务器-->", addr)
    gevent.spawn(fn, conn)
sk.close()
# 输出结果
已连接服务器--> ('127.0.0.1', 53878)
已连接服务器--> ('127.0.0.1', 53879)
已连接服务器--> ('127.0.0.1', 53880)
服务的收到>>> client1
服务的收到>>> client2
服务的收到>>> client3

clien

import socket
sk = socket.socket()
sk.connect(("127.0.0.1", 8899))
msg = input("客户端发送的内容>>> ")
sk.send(msg.encode("utf-8"))
msg = sk.recv(1024).decode("utf-8")
print("客户端收到>>>", msg)
sk.close()
# 输出结果
客户端发送的内容>>> client1
客户端收到>>> client1

The above is the detailed content of Detailed explanation of python coroutines (with examples). For more information, please follow other related articles on the PHP Chinese website!