Default scope and closures in Python and Lua

Default scope

I learned Lua some time ago and found that the default scope of Lua is opposite to that of Python. When Lua defines a variable, the default scope of the variable is global. This is not very accurate. When Lua executes a statement such as x = 1, it will search for x layer by layer starting from the current environment. Only when it cannot find x Global variables are defined only under certain circumstances), and when Python defines a variable, the scope of the variable is local (current block) by default. In addition, Lua can define local variables by adding the local keyword before the variable when defining the variable, but Python does not have a similar keyword, and Python variables can only be defined in the current block.

We know that global variables are bad, but local variables are good. When writing programs, you should try to use local variables. So at the beginning, I thought Python's convention was better. Its advantage is that it requires less typing. When writing Lua programs, I keep saying "Don't forget local, don't forget local" in my heart. However, sometimes a few things slip through the net and cause some magical bugs.

Closures

The first time I realized the problem of Python's default scope was when using closures. Regarding closures, there is a piece of code in the Lua tutorial:

function new_counter()
  local n = 0
  local function counter()
    n = n + 1
    return n
  end
  return counter
end
   
c1 = new_counter()
c2 = new_counter()
print(c1())  -- 打印1
print(c2())  -- 打印1
print(c1())  -- 打印2
print(c2())  -- 打印2

The essence of closures can be referred to the environment model in Chapter 3 of SICP. Here you can simply imagine that the function counter has a private member n.

Now the question comes: I want to use Python to implement a closure with the same function?

First, directly rewrite the Lua code into Python code:

def new_counter():
  n = 0
  def counter():
    n = n + 1
    return n
  return counter

Then I was dumbfounded: This program cannot run, and the unassigned variable n is accessed in line 4. The reason for the error is not that Python does not support closures, but that Python’s assignment operation cannot access the upper-level variable n (in fact, Python considers it to be a definition of local variables, not an assignment. Defining local variables and assignment operations in Python Syntactically conflicting, Python simply supports only redefinable definition statements). Since Python's default scope is local, when the program runs to n = n + 1, Python thinks this is a variable definition operation, so it creates an (uninitialized) local variable n - and successfully overwrites new_counter n at this level - then try to assign n + 1 to n, but n is not initialized, n + 1 cannot be calculated, so the program reports an error.

You can use a little trick to implement the function of closure assignment:

def new_counter():
  n = [0]
  def counter():
    n[0] = n[0] + 1
    return n[0]
  return counter

The reason why n[0] = n[0] + 1 will not go wrong is that the equal sign here and the preceding n The equal sign of = n + 1 has different meaning. The equal sign in n[0] = n[0] + 1 means to modify an attribute of n. In fact, this equal sign ultimately calls the __setitem__ method of list. The equal sign in n = n + 1 means to bind the value n + 1 to symbol n in the current environment (if symbol n already exists in the current environment, overwrite it).

Another digression: Damn it, I don’t need to write it this way, it’s so ugly. Anyway, Python is an object-oriented language. To implement a counter, you might as well write a class.

Separation of definition and assignment

First, let’s summarize the characteristics of the default scope of Python and Lua:

1. Lua’s default scope is global. When writing a program, remember the local keyword (unless you really want to define a global variable) , if you accidentally forget local, there will be no prompt, just wait for the bug to be corrected.

2. Python’s default scope is local. Although the mental burden of writing a program is less, it loses the ability to assign values ​​to upper-level variables (it can be changed, but it will make the language more confusing).

It seems that both default scopes have problems? Personally, I think the reason for the above problems is that Python and Lua do not realize the separation of definition and assignment. In Python and Lua, a statement like x = 1 can represent either a definition or an assignment. In fact, not only these two languages, but many high-level languages ​​do not realize the separation of definition and assignment. Definition and assignment are similar in function, but they are essentially different.

The following uses x = 1 as an example to explain the definition and assignment:

The definition means: register the symbol x in the current environment and initialize it to 1. If x already exists, an error is reported (redefinition is not allowed) or overwritten (redefinition is allowed).

Assignment means: starting from the current environment, looking up layer by layer until the symbol x is found for the first time, and changing its value to 1. If it cannot be found, an error will be reported (the variable does not exist).

Now we slightly modify Python to achieve the separation of definition and assignment: use ":=" to indicate definition and "=" to indicate assignment. Then rewrite the new_counter example that cannot run (the assignment operation in Python conflicts with the definition of local variables. In other words, Python actually does not have an assignment operation, so we only need to simply replace all "=" with ":="") , see where it goes wrong:

def new_counter():
  n := 0
  def counter():
    n := n + 1
    return n
  return counter

It’s obvious why this program is wrong. What we want in line 4 is an assignment operation, not a definition operation. Modify it to the correct writing:

def new_counter():
  n := 0
  def counter():
    n = n + 1
    return n
  return counter

This way it can run correctly (provided there is a modified version of the Python interpreter XD).

Finally, let’s talk about Lua. Lua feels like half of the separation of definition and assignment has been achieved. The equal sign statement with the local keyword must be defined. The problem is the equal sign statement without local. For this kind of statement, Lua does this: first try to do the assignment, and if the assignment fails (the variable does not exist), define the variable in the outermost environment (global environment). In other words, the equal sign statement without local mixes definition and assignment. In addition, if the separation of definition and assignment is achieved, there is no need to consider the issue of default scope - all definitions are defined in the current environment, and they all define local variables. I really can't think of any benefit to defining global variables in a function body or some other block.