The with statement and context manager in Python

As a context manager object in Python, you can use the with statement. The use of the contextlib module that provides the context manager is an advanced skill in Python programming. Let’s sort out the with statement and context management in Python in detail. Machine learning summary:

0. About the context manager
The context manager is an object that can be used in the with statement and has __enter__ and __exit__ methods.

with manager as var:
  do_something(var)

is equivalent to a simplification of:

var = manager.__enter__()
try:
  do_something(var)
finally:
  manager.__exit__()

In other words, in PEP 343 The defined context manager protocol allows the boring try...except...finally structure to be abstracted into a separate class, leaving only the do_something part of concern.

__enter__ method is called first. It can return the value assigned to var. The as part is optional: if it does not appear, the return value of enter is simply ignored.
The code under the with statement is executed. Like try clauses, they either execute successfully, break, continue, or return, or they can throw an exception. In either case, after the block ends, the __exit__ method is called. If an exception is thrown, the exception information is passed to __exit__, which is discussed in the next section. Normally, exceptions can be ignored, as in a finally clause, and will be rethrown after __exit__ ends.
For example, we want to make sure that a file is closed immediately after completing the write operation:

>>> class closing(object):
...  def __init__(self, obj):
...   self.obj = obj
...  def __enter__(self):
...   return self.obj
...  def __exit__(self, *args):
...   self.obj.close()
>>> with closing(open('/tmp/file', 'w')) as f:
...  f.write('the contents\n')

Here we ensure that f is called when the with block exits .close(). Because closing a file is a very common operation, this support is already present in the file class. It has an __exit__ method called close and acts as a context manager itself.

>>> with open('/tmp/file', 'a') as f:
...  f.write('more contents\n')

try... A common usage of finally is to release resources. The various situations are implemented similarly: resources are acquired in the __enter__ phase, released in the __exit__ phase, and if an exception is thrown, it is also passed on. Just like file operations, this is often a natural operation after using the object, and built-in support makes it convenient. With each version, Python is supported in more places.

1. How to use the context manager:

How to open a file and write "hello world"

filename="my.txt"
mode="w"
writer=open(filename,mode)
writer.write("hello world")
writer.close()

When an exception occurs (such as the disk is full), there is no chance to execute line 5. Of course, we can use a try-finally statement block for packaging:

writer=open(filename,mode)
try:
  writer.write("hello world")
finally:
  writer.close()

When we perform complex operations, the try-finally statement will become ugly, Rewrite using the with statement:

with open(filename,mode) as writer:
  writer.write("hello world")

as refers to the content returned from the open() function and assigns it to the new value. with completes the task of try-finally.

2. Custom context manager

The function of the with statement is similar to try-finally, providing a context mechanism. To apply the with statement, the class must provide two built-in functions __enter__ and __exit__. The former is executed before the main code is executed, and the latter is executed after the main code is executed. The variables after as are returned in the __enter__ function.

class echo():
  def output(self):
    print "hello world"
  def __enter__(self):
    print "enter"
    return self #可以返回任何希望返回的东西
  def __exit__(self,exception_type,value,trackback):
    print "exit"
    if exception_type==ValueError:
      return True
    else:
      return Flase
 
>>>with echo as e:
  e.output()

Output:

##

enter
hello world
exit

Complete__exit_ _The function is as follows:

def __exit__(self,exc_type,exc_value,exc_tb)

Among them, exc_type: exception type; exc_value: exception value; exc_tb: exception tracking information

When __exit__ When True is returned, the exception is not propagated

3. contextlib module

The role of the contextlib module is to provide an easier-to-use context manager, which is implemented through Generator. The contextmanager in contextlib serves as a decorator to provide a function-level context management mechanism. The commonly used frameworks are as follows:

from contextlib import contextmanager
@contextmanager
def make_context():
  print 'enter'
  try:
    yield "ok"
  except RuntimeError,err:
    print 'error',err
  finally:
    print 'exit'
    
>>>with make_context() as value:
  print value

The output is:

  enter
  ok
  exit

Among them, yield is written into try-finally to ensure exception safety (can handle exceptions). The value of the variable after as is returned by yield. The statement before yield can be regarded as the operation before the code block is executed, and the operation after yield can be regarded as the operation in the __exit__ function.

Take thread lock as an example:

@contextlib.contextmanager
def loudLock():
  print 'Locking'
  lock.acquire()
  yield
  print 'Releasing'
  lock.release()
 
with loudLock():
  print 'Lock is locked: %s' % lock.locked()
  print 'Doing something that needs locking'
 
#Output:
#Locking
#Lock is locked: True
#Doing something that needs locking
#Releasing

4. contextlib.nested: Reduce nesting

For:

with open(filename,mode) as reader:
  with open(filename1,mode1) as writer:
    writer.write(reader.read())

can be simplified by contextlib.nested:

with contextlib.nested(open(filename,mode),open(filename1,mode1)) as (reader,writer):
  writer.write(reader.read())

In python 2.7 and later, it is replaced by a new syntax:

with open(filename,mode) as reader,open(filename1,mode1) as writer:
  writer.write(reader.read())

5, contextlib.closing()

The file class directly supports the context manager API, but some objects representing open handles are not supported, such as the objects returned by urllib.urlopen(). There are also some legacy classes that use the close() method but do not support the context manager API. To ensure that the handle is closed, you need to create a context manager for it using closing() (calling the class's close method).

import contextlib
class myclass():
  def __init__(self):
    print '__init__'
  def close(self):
    print 'close()'
   
with contextlib.closing(myclass()):
  print 'ok'

Output:

__init__
ok
close()

For more articles related to the with statement and context manager in Python, please pay attention to the PHP Chinese website!