Examples to explain issues that should be paid attention to when using Python function closures

Yesterday, when I was poking at the keyboard with 10% of my yang finger power and coding in the dark, I happened to be asked a question, and I almost lost my good work. The ancient power leaked and hurt the people at the table. Nonsense. Without further ado, let’s start with chestnuts (a simplified version, just to illustrate the problem):

from functools import wraps
from time import sleep

def retry(attempts=3, wait=2):
  if attempts < 0 or attempts > 5:
    retry_times = 3
  else:
    retry_times = attempts
  if wait < 0 or wait > 5:
    retry_wait = 2
  else:
    retry_wait = after
  def retry_decorator(func):
    @wraps(func)
    def wrapped_function(*args, **kwargs):
      while retry_times > 0:
        try:
          return func(*args, **kwargs)
        except :
          sleep(retry_wait)
          retry_times -= 1
    return wrapped_function
  return retry_decorator

The simple version of the retry decorator, the required variables are perfectly captured by the closure, and the logic is quite simple and clear. The person who asked said that the logic seemed to be quite normal, but the error message that the variable retry_times could not be found (unresolved reference) kept getting reported.

Yes, check it carefully, this is a scoring question: the variables captured by the closure (retry_times, retry_wait) are equivalent to the local variables of the retry function that are referenced at the time. When the local effect of wrapped_function is used to operate the immutable type, When data is generated, a new local variable will be generated, but the newly generated local variable retry_times has not had time to be initialized before use, so it will prompt that the variable cannot be found; retry_wait can be used properly.

Python is a duck-typing programming language. Even if there is a warning, it will still run. Write a function as simple as possible, use a decorator, and put a breakpoint in the wrapped_function logic to see the value of each variable. It can be done very quickly. Found the problem (you can also see the error by running directly: UnboundLocalError: local variable 'retry_attempts' referenced before assignment, at least more useful than warning msg):

@retry(7, 8)
def test():
  print 23333
  raise Exception('Call me exception 2333.')

if __name__ == '__main__':
  test()

output: UnboundLocalError: local variable 'retry_times' referenced before assignment

To solve this kind of problem is easy, just use a mutable container to wrap the immutable type of data to be used (it’s a completely irresponsible digression to say that I haven’t written C# code for a long time and can’t remember it clearly) , just like in C#.net, when a closure is encountered, a class with an obfuscated name will be automatically generated and the value to be captured will be stored as an attribute of the class, so that it can be easily obtained when using it. The famous Lao Zhao seems to have an article about Lazy Evaluation that seems to involve this topic):

def retry(attempts=3, wait=2):
  temp_dict = {
    'retry_times': 3 if attempts < 0 or attempts > 5 else attempts,
    'retry_wait': 2 if wait < 0 or wait > 5 else wait
  }

  def retry_decorate(fn):
    @wraps(fn)
    def wrapped_function(*args, **kwargs):
      print id(temp_dict), temp_dict
      while temp_dict.get('retry_times') > 0:
        try:
          return fn(*args, **kwargs)
        except :
          sleep(temp_dict.get('retry_wait'))
          temp_dict['retry_times'] = temp_dict.get('retry_times') - 1
        print id(temp_dict), temp_dict

    print id(temp_dict), temp_dict

    return wrapped_function

  return retry_decorate

@retry(7, 8)
def test():
  print 23333
  raise Exception('Call me exception 2333.')

if __name__ == '__main__':
  test()

Output:

4405472064 {'retry_wait': 2, 'retry_times': 3}
4405472064 {'retry_wait': 2, 'retry_times': 3}
23333
4405472064 {'retry_wait': 2, 'retry_times': 2}
23333
4405472064 {'retry_wait': 2, 'retry_times': 1}
23333
4405472064 {'retry_wait': 2, 'retry_times': 0}

As you can see from the output, after wrapping it with dict, the program can work normally, as expected. In fact, we can also confirm again from the value of the closure of the function:

>>> test.func_closure[1].cell_contents
{'retry_wait': 2, 'retry_times': 2}

I am the ending, PEACE!