Analysis of variable references in python closures

The title looks very tigerish, but in fact I dare not call it analysis. I still have some shortcomings in this area. It might have been OK before, but now I don’t have as much time and energy to study languages. If there are any incorrect explanations, you are welcome to come to Banzhu. Don’t mislead the public.

Let’s talk about the problem directly this time. Today @neiddy(javaeye) talked to me about the problem of closure. It was very interesting to see those few examples, and I wanted to understand it.

Look at two pieces of code:

>>> def foo():
       a = 1
       def bar():
              a = a +1
              return a
       return bar()
 
>>> foo()
Traceback (most recent call last):
  File "<pyshell#73>", line 1, in <module>
    foo()
  File "<pyshell#72>", line 6, in foo
    return bar()
  File "<pyshell#72>", line 4, in bar
    a = a +1
UnboundLocalError: local variable &#39;a&#39; referenced before assignment

>>> def foo():
       a = [1]
       def bar():
              a[0] = a[0] + 1
              return a[0]
       return bar()
 
>>> foo()2

It feels good to experience functional programming through closures. There are no parentheses below the original text, otherwise the return function itself would be meaningless. Let’s talk about this magical phenomenon. The article just says that it will be ok if it is changed to a container. Why?

I Googled some things about memory management and memory allocation, but I couldn't find a way out. I had no choice but to turn to the source code. I have never seen it before and it is too inefficient to read it directly. Fortunately, someone has summarized it and recommends "In-depth Python Programming" by "Yuhen Q.yuhen". He is really a low-key master.

The code analysis of the direct closure part is mentioned in the section on closures, but there is no mechanism to specifically talk about this strange problem.

Read the book again and refer to the 'Parameters' and 'Closure' sections in the book. Let me talk about my understanding based on his explanation and the attached source code.

Yu Chen mentioned at the end of the chapter, "The principle of CPython implementing closure is not complicated. To put it bluntly, it is to attach the referenced outer object to the inner function object (func_closure) that is recreated every time."

This sentence is a summary of the previous one, and it also contains important information, that is, the object corresponding to the inner function accesses the variables in the outer function by referencing the outer variable to the stack of the memory object. In C language code, it is passed by value.

For example, a in the first code actually refers to 1, and its own co_nlocals is 1, that is, a local variable is the a in front of the equal sign (this is not quite right, but I just hope it helps to understand this problem). Since it is a local variable a, a = a +1 must throw UnboundLocalError.

As for the second question, although the same situation exists, even if it is passed by value, the pointer pointing to each position in the array will not change, and the value of the specified position will still be modified. Therefore, if it is a container object It just works.

What I hate about myself is that I always feel like I can’t explain clearly. In fact, it’s a pointer game that I often played when I first learned C language. Although the explanation is terrible, I hope it has pointed out the key point. The next step is very simple. Follow this idea to verify a piece of code. That is, if you just output this value and do not set local variables, it should be able to run.

>>> def foo():
       a = 1
       def bar():
              return a
       return bar()
 
>>> foo()
1

Things happened as expected. To experience it more carefully, take a good look at the code analyzed by Yuhen. Understanding the mechanics goes a long way. Python is becoming more and more interesting. Writing code with python ideas has many benefits.

By the way, by the way

>>> flist = []
>>> for i in range(3):
       def foo(x): print x + i
       flist.append(foo)
 
>>> for f in flist:
       f(2)
 
4
4
4

The article mentioned that this code is caused by i not being destroyed. I also saw this thing when I searched for memory allocation today. When programming, you should try to use list comprehension to reduce for and while. On the one hand, functional programming is concise and clear, on the other hand, it improves performance and saves a counter.