Detailed explanation of yield from syntax in Python 3

In python 3.3, generator has a new syntax yield from. What is the function of yield from? What is the syntax? The following article mainly introduces the relevant information of yield from syntax in Python 3 to you in detail. Friends who need it can refer to it. Let’s take a look together.

Preface

I was tinkering with Autobahn recently, and it gave an example based on asyncio. I thought about putting it on pypy3 and running it, but it... failed. . pip install asyncio reported invalid syntax directly. At first glance, I thought there was a problem with 2to3 processing - you can't blame me. Well, many packages were written in 2 and then converted to 3 - but it turned out that asyncio Originally it only supported version 3.3+, so I looked back at the code and suddenly found the sentence yield from; yield I know, but yield from is a magic horse?

PEP-380

Okay, I came up with this title through Google. yield from’s past and present lives are all in this PEP. In short, the general idea is The original yield statement can only return CPU control to the direct caller. When you want to reconstruct the logic with a yield statement in a generator or coroutine into another generator (original text is subgenerator) It will be very troublesome because the outside generator is responsible for message passing for the inside generator; so someone had the idea to let python encapsulate the message passing to make it transparent to programmers, so there is yield from.

PEP-380 specifies the semantics of yield from, or the behavior pattern that nested generators should have.

Suppose there is such a statement in function A

yield from B()

##B() returns an iterable (iterable ) object b, then A() will return a generator - according to our naming convention, the name is a - then:

1. b Each value generated by iteration is passed directly to the caller of a.

2. All values ​​sent to a through the send method are passed directly to b. If the value sent is None, the

   __next__() method of b is called, Otherwise, call the send method of b. If the method call to b generates a StopIteration exception, a will continue to execute the statements following yield from, and other exceptions will be propagated to a, causing a to execute yield from throw an exception.

3. If an exception other than GeneratorExit is thrown into a, the exception will be thrown directly into b. If b's throw method throws StopIteration, a will continue to execute; other exceptions will cause a to also throw an exception.

4. If a GeneratorExit exception is thrown into a, or a's close method is called, and b also has a close method, b's close method will also be called. If this method of b throws an exception, it will cause a to also throw an exception. On the contrary, if b is successfully closed, a will also throw an exception, but it is a specific GeneratorExit exception.

5. The evaluation result of the

   yield from expression in a is the first parameter of the StopIteration exception thrown at the end of the iteration of b.

6. The

   return 942bcdedd51e2f7d833cc5298f53fcca statement in b will actually throw a StopIteration(942bcdedd51e2f7d833cc5298f53fcca) exception, so in b The value of return will become the return value of the yield from expression in a.
   

Why does Shenma have so many requirements? Because the behavior of generators becomes very complicated after adding the throw method, especially when several generators are together, primitives similar to process management are needed to operate them. All the above requirements are to unify the inherently complex behavior of the generator, so naturally it cannot be simplified.

I admit that I didn’t understand what the author of the PEP wanted to say, so it might be helpful to “refactor” it.

A useless example

It’s useless because you probably don’t really want to write the program like this, but... Anyway, it’s enough to illustrate the problem. .

Imagine there is such a generator function:

def inner():
 coef = 1
 total = 0
 while True:
 try:
  input_val = yield total
  total = total + coef * input_val
 except SwitchSign:
  coef = -(coef)
 except BreakOut:
  return total

The generator generated by this function will accumulate the value received from the send method to the local variable total , and stop iteration when receiving a BreakOut exception; as for the other SwitchSign exception, it should not be difficult to understand, so I won’t spoil it here.

From the code point of view, the generator obtained by the

inner() function receives data for calculation through send, and at the same time accepts the control of external code through the throw method to execute different code branches. So clear so far.

Next, due to changes in requirements, we need to add initialization and site cleanup code before and after

inner(). Since I think "if the code is not broken, don't touch it", I decided to keep inner() as it is, and then write an outer() and put the added code in outer(), and provides the same operation interface as inner(). Since inner() utilizes several features of generator, outer() must also do these five things:

1. outer()必须生成一个generator；

2. 在每一步的迭代中，outer()要帮助inner()返回迭代值；

3. 在每一步的迭代中，outer()要帮助inner()接收外部发送的数据；

4. 在每一步的迭代中，outer()要处理inner()接收和抛出所有异常；

5. 在outer()被close的时候，inner()也要被正确地close掉。
   

根据上面的要求，在只有yield的世界里，outer()可能是长这样的：

def outer1():
 print("Before inner(), I do this.")
 i_gen = inner()
 input_val = None
 ret_val = i_gen.send(input_val)
 while True:
 try:
  input_val = yield ret_val
  ret_val = i_gen.send(input_val)
 except StopIteration:
  break
 except Exception as err:
  try:
  ret_val = i_gen.throw(err)
  except StopIteration:
  break
 print("After inner(), I do that.")

WTF，这段代码比inner()本身还要长，而且还没处理close操作。

现在我们来试试外星科技：

def outer2():
 print("Before inner(), I do this.")
 yield from inner()
 print("After inner(), I do that.")

除了完全符合上面的要求外，这四行代码打印出来的时候还能省点纸。

我们可以在outer1()和outer2()上分别测试 数据 以及 异常 的传递，不难发现这两个generator的行为基本上是一致的。既然如此， 外星科技当然在大多数情况下是首选。

对generator和coroutine的疑问

从以前接触到Python下的coroutine就觉得它怪怪的，我能看清它们的 行为模式，但是并不明白为什么要使用这种模式，generator和 coroutine具有一样的对外接口，是generator造就了coroutine呢，还 是coroutine造就了generator？最让我百思不得其解的是，Python下 的coroutine将“消息传递”和“调度”这两种操作绑在一个yield 上——即便有了yield from，这个状况还是没变过——我看不出这样做 的必要性。如果一开始就从语法层面将这两种语义分开，并且为 generator和coroutine分别设计一套接口，coroutine的概念大概也会 容易理解一些。

更多Python 3中的yield from语法详解相关文章请关注PHP中文网！