Deep understanding of coroutine functions in python

The content of this article is to share with you an in-depth understanding of the coroutine function in python, which has a certain reference value. Friends in need can refer to it

Concept:

According to the definition given by Wikipedia, "Coroutines are computer program components that generate subroutines for non-preemptive multitasking. Coroutines allow different entry points to pause or start program execution at different locations." From a technical perspective, "a coroutine is a function that you can pause execution of." If you understand it as "just like a generator", then you're thinking right.

Coroutine, also known as micro-thread, looks like a subroutine, but it is different from a subroutine. During the execution process, it can interrupt the current subroutine. After the program executes other subroutines, it returns to execute the previous subroutine, but its related information is still the same as before.

Coroutines are different from threads. Threads are preemptive scheduling, while coroutines are collaborative scheduling. Coroutines need to do their own scheduling.
Subroutine calls always have one entrance and one return, and the calling sequence is clear. The calling of coroutines is different from subroutines. Coroutines also look like subprograms, but during execution, they can be interrupted inside the subprogram, then switch to executing other subprograms, and then return to continue execution at the appropriate time.

Advantages of coroutines:

• The advantage of coroutines is extremely high execution efficiency. Because subroutine switching is not thread switching, but is controlled by the program itself, there is no overhead of thread switching. Compared with multi-threading, the greater the number of threads, the more obvious the performance advantages of coroutines. It is very suitable for performing coroutine multitasking.

• Coroutines have no thread safety issues. A process can have multiple coroutines at the same time, but only one coroutine is active, and the activation and dormancy of coroutines are controlled by programmers through programming, not by the operating system.

Generator implements coroutine principle

Example:

def func(n):
    index=0
    if index<=n:
        c=yield 1
        print("task------{}".format(c))
        index+=1f=func(3)
n=next(f)
print(n)try:
    n=f.send(5)#程序就直接结束了
    print("n是{}".format(n))except StopIteration as e:    pass

输出打印：1task------5

Explanation:

• Obviously func is a generator, and the send method has a parameter that specifies the return value of the last suspended yield statement.

• send requires exception handling.

• In general, the only difference between the send method and the next method is that when executing the send method, the return value of the last pending yield statement will first be set through parameters to achieve Interaction with generator methods. However, it should be noted that before a generator object executes the next method, since no yield statement is suspended, an error will be reported when executing the send method.

• When the parameter of the send method is None, it is completely equivalent to the next method.

The generator implements the producer and consumer patterns:

def cunsumer():
    while True:
        n=yield 3
        if not n:            return
        print('cunsumer{}'.format(n))def product(c):
    c.send(None)
    n=0
    while n<5:
        n=n+1
        r=c.send(n)
        print("product{}".format(r))
    c.close()
c=cunsumer()
product(c)

打印：
cunsumer1
product3
cunsumer2
product3
cunsumer3
product3
cunsumer4
product3
cunsumer5
product3

Explanation:

Execute first in the producer The purpose of c.send(None) is to let the consumer hang up first, and then use send to pass the value. The first time 1 is passed, the consumer prints 1, and the producer prints r which is the value after the consumer's yield.

Introduction of greenlet

Although CPython (standard Python) can implement coroutines through generators, it is not very convenient to use.

At the same time, Stackless Python, a derivative of Python, implements native coroutines, which is more convenient to use.

So, everyone began to take out the coroutine code in Stackless and make it into a CPython expansion package.

This is the origin of greenlet, so greenlet is a C extension library that implements native coroutines at the bottom.

Code description:

from greenlet import greenletimport randomimport timedef Producer():
    while True:
        item = random.randint(0,10)
        print("生产了{}".format(item))
        c.switch(item)#切换到消费者，并将item传入消费者
        time.sleep(1)def consumer():
    print('我先执行')    #p.switch()
    while True:
        item = p.switch()#切换到生产者，并且等待生产者传入item
        print('消费了{}'.format(item))
c = greenlet(consumer)#将一个普通函数变成一个协程p = greenlet(Producer)
c.switch()#让消费者先进入暂停状态（只有恢复了才能接收数据）

Value of greenlet:

• High-performance native coroutine

• Explicit switching with clearer semantics

• Wrap functions directly into coroutines and maintain the original code style

gevent coroutine

Although we have a callback programming model based on epoll, it is difficult to use.

Even though we can make complex encapsulation with generator coroutine to simplify programming difficulty.
But there is still a big problem: encapsulation is difficult, and the existing code must be almost completely rewritten

gevent, by encapsulating the two libraries libev (based on epoll) and greenlet.
Help us encapsulate it and allow us to use coroutines in a thread-like manner.

So that we can make full use of the power of epoll and coroutines without rewriting the original code.

Code diagram:

from gevent import monkey;monkey.patch_all()#会把python标准库当中一些阻塞操作变成非阻塞import geventdef test1():
    print("11")
    gevent.sleep(4)#模拟爬虫请求阻塞
    print("33")def test2():
    print("22")
    gevent.sleep(4)
    print("44")
gevent.joinall([gevent.spawn(test1),gevent.spawn(test2)])#joinall 阻塞当前协程，执行给定的greenlet#spawn 启动协程，参数就是函数的名字

The value of gevent:

Switch to another coroutine to continue execution when it encounters blocking !

• Use epoll-based libev to avoid blocking.

• Use efficient coroutines based on gevent to switch execution.

• Only switches when encountering blocking. There is no round-robin overhead or thread overhead.

gevent implements concurrent server

from gevent import monkey;monkey.patch_all()  #建议放在首行，会把python标准库当中一些阻塞操作变成非阻塞import geventimport socket


server=socket.socket()
server.bind(('',6666))
server.listen(5)
print("开始监听")def readable(con,addr):
    print("客户端{}接入".format(addr))    while True:
        data=con.recv(1024)        if data:
            print(data)        else:
            con.close()            breakwhile True:
    con,addr=server.accept()
    gevent.spawn(readable,con,addr)#将readable函数变为协程，并且把con和addr传入其中。

gevent coroutine communication

gevent also has its own queue. The usage is basically the same as threading.

Producer and consumer patterns based on gevent and queue

from gevent import monkey;monkey.patch_all()import geventfrom gevent.queue import Queueimport randomdef producter(queue):
    while True:
        item=random.randint(0,99)
        print('生产了{}'.format(item))
        queue.put(item)
        gevent.sleep(1)def comuser(queue):
    while True:
        item=queue.get()
        print('消费了{}'.format(item))
queue=Queue()
p=gevent.spawn(producter,queue)
c=gevent.spawn(comuser,queue)
gevent.joinall([p,c])

打印：
生产了33消费了33生产了95消费了95生产了92消费了92...

相关推荐:

python中多进程+协程的使用

python中协程

Python 协程的详细用法和例子

python 协程示例

The above is the detailed content of Deep understanding of coroutine functions in python. For more information, please follow other related articles on the PHP Chinese website!