Detailed explanation of coroutines in python (with examples)

This article brings you a detailed explanation of coroutines in Python (with examples). It has certain reference value. Friends in need can refer to it. I hope it will be helpful to you.

Coroutines, also known as micro-threads and fibers. The English name Coroutine
Coroutine looks like a subprogram, but during execution, it can be interrupted inside the subprogram, and then switch to executing other subprograms, and then return to continue execution at the appropriate time.

The biggest advantage is the extremely high execution efficiency of coroutines. Because subroutine switching is not thread switching, but is controlled by the program itself, there is no overhead of thread switching. Compared with multi-threading, the greater the number of threads, the more obvious the performance advantages of coroutines.
The second biggest advantage is that there is no need for a multi-threaded lock mechanism, because there is only one thread, and there is no conflict in writing variables at the same time. Shared resources are controlled in the coroutine without locking, and only the status needs to be determined, so execute The efficiency is much higher than multi-threading.
Because the coroutine is executed by a thread, how to use multi-core CPU? The simplest method is multi-process coroutine, which not only makes full use of multiple cores, but also gives full play to the high efficiency of coroutine, and can achieve extremely high performance.

yield implements coroutines

Python's support for coroutines is still very limited. Yield used in generators can implement coroutines to a certain extent. Although the support is not complete, it can already exert considerable power.

import threading
import time
def producer(c):
    c.__next__()
    n=0
    while n c.send(n) --> n更新
        n = yield r
        if not n:
            break
        print('[消费者]正在调用第%s条数据' %(n))
        time.sleep(1)
        r = 'This is ok!'

if __name__=='__main__':
    print(threading.current_thread())   
    print(threading.active_count())     #查看当前进行的线程
    c = consumer()
    producer(c)     #函数中有yield， 返回值为生成器;
    print(threading.active_count()) #1

gevent library implements coroutines

Python provides basic support for coroutines through yield , but not completely. The third-party gevent provides relatively complete coroutine support for Python.

gevent is a third-party library that implements coroutines through greenlets. The basic idea is:
When a greenlet encounters an IO operation, such as accessing the network, it automatically switches to other greenlets, waits until the IO operation is completed, and then Switch back to continue execution when appropriate. Since IO operations are very time-consuming, the program is often placed in a waiting state. With gevent automatically switching coroutines for us, it is guaranteed that greenlets are always running instead of waiting for IO.

Since switching is automatically completed during IO operations, gevent needs to modify some of the standard libraries that come with Python. This process is completed through monkey patch at startup.

Assuming that the tasks executed by multiple coroutines have no IO operations or waiting, then the coroutines run sequentially instead of alternately;
Assuming that the tasks executed by multiple coroutines have IO operations or waiting, then Coroutines run alternately;

#没有等待
import gevent
from gevent import monkey
monkey.patch_all()
def job(n):
    for i in range(n):
        print(gevent.getcurrent(),i)

def mian():
    g1 = gevent.spawn(job,1)
    g2 = gevent.spawn(job,2)
    g3 = gevent.spawn(job,3)
    gevent.joinall([g1,g2,g3])
    print('协程执行任务结束...')

if __name__=="__main__":
    mian()

"""
#有等待
import time
from gevent import  monkey
monkey.patch_all()

import  gevent
def job(n):
    for i in range(n):
        print(gevent.getcurrent(), i)
        time.sleep(1)

def main1():
    # 创建三个协程， 并让该协程执行job任务
    g1 = gevent.spawn(job, 2)
    g2 = gevent.spawn(job, 3)
    g3 = gevent.spawn(job, 2)
    # 等待所有的协程执行结束， 再执行主程序；
    gevent.joinall([g1, g2, g3])
    print("任务执行结束.....")

main1()

Coroutines Do a comparative experiment on the time spent by coroutines and threads with threads

, which is not informative.

import time
import gevent   #导入协程
from gevent import monkey
from urllib.request import urlopen  #连接网络
from mytimeit import timeit #导入计算时间的装饰器
from concurrent.futures import ThreadPoolExecutor   #导入线程池

def get_len_url(url):
    with urlopen(url) as u_conn:
        data = u_conn.read()
#       print('%s该网页共%s字节' %(url,len(data)))
urls = ['http://httpbin.org', 'http://example.com/']*100

@timeit
def coroutineall():
    gevents = [gevent.spawn(get_len_url,url) for url in urls]
    gevent.joinall(gevents)

@timeit
def threadall():
    with ThreadPoolExecutor(max_workers=100) as thpool:
        thpool.map(get_len_url,urls)
if __name__=="__main__":
    coroutineall()
    threadall()

The above is the detailed content of Detailed explanation of coroutines in python (with examples). For more information, please follow other related articles on the PHP Chinese website!