Detailed explanation of Python's decorators

The decorator in Python is a hurdle for you to enter Python. It is there whether you cross it or not.

Why do you need decorators

We assume that your program implements the two functions say_hello() and say_goodbye().

def say_hello(): 
    print "hello!" 
     
def say_goodbye(): 
    print "hello!"  # bug here 
 
if __name__ == '__main__': 
    say_hello() 
    say_goodbye()

But in the actual call, we found that the program went wrong, and the above code printed two hellos. After debugging, you find that there is an error in say_goodbye(). The boss requires that the name of the entering function be recorded before calling each method, such as this:

[DEBUG]: Enter say_hello() 
Hello! 
[DEBUG]: Enter say_goodbye() 
Goodbye!

Okay, little A is a graduate, and he implemented it like this.

def say_hello(): 
    print "[DEBUG]: enter say_hello()" 
    print "hello!" 
 
def say_goodbye(): 
    print "[DEBUG]: enter say_goodbye()" 
    print "hello!" 
 
if __name__ == '__main__': 
    say_hello() 
    say_goodbye()

Very low, right? Yes. Little B has been working for a while, and he told Little A that he could write like this.

def debug(): 
    import inspect 
    caller_name = inspect.stack()[1][3] 
    print "[DEBUG]: enter {}()".format(caller_name)    
 
def say_hello(): 
    debug() 
    print "hello!" 
 
def say_goodbye(): 
    debug() 
    print "goodbye!" 
 
if __name__ == '__main__': 
    say_hello() 
    say_goodbye()

Is it better? Of course, but it is uncomfortable to call the debug() function in every business function? What if the boss says that debugging is not needed for say-related functions, but only for do-related functions?

Then the decorator should appear at this time.

A decorator is essentially a Python function, which allows other functions to add additional functions without making any code changes. The return value of the decorator is also a function object. It is often used in scenarios with cross-cutting requirements, such as: log insertion, performance testing, transaction processing, caching, permission verification, etc. Decorators are an excellent design to solve such problems. With decorators, we can extract a large amount of similar code that has nothing to do with the function itself and continue to reuse it.

In a nutshell, the role of a decorator is to add additional functionality to an existing function or object.

How to write a decorator

In the early days (Python Version

def debug(func): 
    def wrapper(): 
        print "[DEBUG]: enter {}()".format(func.__name__) 
        return func() 
    return wrapper 
 
def say_hello(): 
    print "hello!" 
 
say_hello = debug(say_hello)  # 添加功能并保持原函数名不变

The debug function above is actually a decorator. It wraps the original function and returns another function, adding some additional functions. Because writing this way is not very elegant, later versions of Python support @ syntactic sugar. The following code is equivalent to the earlier way of writing.

def debug(func): 
    def wrapper(): 
        print "[DEBUG]: enter {}()".format(func.__name__) 
        return func() 
    return wrapper 
 
@debug 
def say_hello(): 
    print "hello!"

This is the simplest decorator, but there is a problem. If the decorated function needs to pass in parameters, then this decorator will break. Because the returned function cannot accept parameters, you can specify that the decorator function wrapper accepts the same parameters as the original function, such as:

def debug(func): 
    def wrapper(something):  # 指定一毛一样的参数 
        print "[DEBUG]: enter {}()".format(func.__name__) 
        return func(something) 
    return wrapper  # 返回包装过函数 
 
@debug 
def say(something): 
    print "hello {}!".format(something)

In this way, you solve one problem, but have N more problems. Because there are thousands of functions, you only care about your own function. Who knows what other people’s function parameters look like? Fortunately, Python provides variable parameters *args and keyword parameters **kwargs. With these two parameters, The decorator can be used for any target function

def debug(func): 
    def wrapper(*args, **kwargs):  # 指定宇宙无敌参数 
        print "[DEBUG]: enter {}()".format(func.__name__) 
        print 'Prepare and say...', 
        return func(*args, **kwargs) 
    return wrapper  # 返回 
 
@debug 
def say(something): 
    print "hello {}!".format(something)

At this point, you have fully mastered the basic method of writing decorators.

Advanced decorators

Decorators with parameters and class decorators are advanced content. Before understanding these decorators, it is best to have a certain understanding of function closures and decorator interface conventions. (See http://betacat.online/posts/p...

Decorator with parameters

Assume that the function that our previous decorator needs to complete is not only to print out log information after entering a certain function, but also You also need to specify the log level, then the decorator will look like this.

def logging(level): 
    def wrapper(func): 
        def inner_wrapper(*args, **kwargs): 
            print "[{level}]: enter function {func}()".format( 
                level=level, 
                func=func.__name__) 
            return func(*args, **kwargs) 
        return inner_wrapper 
    return wrapper 
 
@logging(level='INFO') 
def say(something): 
    print "say {}!".format(something) 
 
# 如果没有使用@语法，等同于 
# say = logging(level='INFO')(say) 
 
@logging(level='DEBUG') 
def do(something): 
    print "do {}...".format(something) 
 
if __name__ == '__main__': 
    say('hello') 
    do("my work")

Are you a little confused? You can understand it this way, when a decorator with parameters is placed on a certain function, such as @logging(level) ='DEBUG'), it is actually a function and will be executed immediately. As long as the result it returns is a decorator, there is no problem. Let’s take a closer look at the class-based decorator

. A decorator function is actually an interface constraint that must accept a callable object as a parameter and then return a callable object. In Python, callable objects are generally functions, but there are exceptions as long as an object overloads __call__(). method, then the object is callable.

class Test(): 
    def __call__(self): 
        print 'call me!' 
 
t = Test() 
t()  # call me

Methods with underscores before and after __call__ are called built-in methods in Python, and are sometimes called magic methods. Overloading these magic methods generally changes the object. The above example allows a class object to have the called behavior.

Back to the concept of the decorator, the decorator requires a callable object and returns a callable object (not too rigorous, see Later). Then it is also possible to implement it with a class. We can let the class's constructor __init__() accept a function, and then overload __call__() and return a function, which can also achieve the effect of a decorator function.

class logging(object): 
    def __init__(self, func): 
        self.func = func 
 
    def __call__(self, *args, **kwargs): 
        print "[DEBUG]: enter function {func}()".format( 
            func=self.func.__name__) 
        return self.func(*args, **kwargs) 
@logging 
def say(something): 
    print "say {}!".format(something)

Class decorator with parameters

If you need to implement a decorator with parameters in a class form, it will be a little more complicated than the previous example. Then what is accepted in the constructor is not a function, but passed in. Parameters. Save these parameters through the class. Then when overloading the __call__ method, you need to accept a function and return a function.

class logging(object): 
    def __init__(self, level='INFO'): 
        self.level = level 
         
    def __call__(self, func): # 接受函数 
        def wrapper(*args, **kwargs): 
            print "[{level}]: enter function {func}()".format( 
                level=self.level, 
                func=func.__name__) 
            func(*args, **kwargs) 
        return wrapper  #返回函数 
 
@logging(level='INFO') 
def say(something): 
    print "say {}!".format(something)

Built-in decorators are the same as ordinary decorators. Yes, but it does not return a function, but a class object, so it is more difficult to understand.

@property

Before understanding this decorator, you need to know how to write a property without using a decorator.

def getx(self): 
    return self._x 
 
def setx(self, value): 
    self._x = value 
     
def delx(self): 
   del self._x 
 
# create a property 
x = property(getx, setx, delx, "I am doc for x property")

以上就是一个Python属性的标准写法，其实和Java挺像的，但是太罗嗦。有了@语法糖，能达到一样的效果但看起来更简单。

@property 
def x(self): ... 
 
# 等同于 
 
def x(self): ... 
x = property(x)

属性有三个装饰器：setter, getter, deleter ，都是在property()的基础上做了一些封装，因为setter和deleter是property()的第二和第三个参数，不能直接套用@语法。getter装饰器和不带getter的属性装饰器效果是一样的，估计只是为了凑数，本身没有任何存在的意义。经过@property装饰过的函数返回的不再是一个函数，而是一个property对象。

>>> property() 
<property object at 0x10ff07940>

@staticmethod，@classmethod

有了@property装饰器的了解，这两个装饰器的原理是差不多的。@staticmethod返回的是一个staticmethod类对象，而@classmethod返回的是一个classmethod类对象。他们都是调用的是各自的__init__()构造函数。

class classmethod(object): 
    """ 
    classmethod(function) -> method 
    """     
    def __init__(self, function): # for @classmethod decorator 
        pass 
    # ... 
class staticmethod(object): 
    """ 
    staticmethod(function) -> method 
    """ 
    def __init__(self, function): # for @staticmethod decorator 
        pass 
    # ...

装饰器的@语法就等同调用了这两个类的构造函数。

class Foo(object): 
 
    @staticmethod 
    def bar(): 
        pass 
     
    # 等同于 bar = staticmethod(bar)

至此，我们上文提到的装饰器接口定义可以更加明确一些，装饰器必须接受一个callable对象，其实它并不关心你返回什么，可以是另外一个callable对象(大部分情况)，也可以是其他类对象，比如property。

装饰器里的那些坑

装饰器可以让你代码更加优雅，减少重复，但也不全是优点，也会带来一些问题。

位置错误的代码

让我们直接看示例代码。

def html_tags(tag_name): 
    print 'begin outer function.' 
    def wrapper_(func): 
        print "begin of inner wrapper function." 
        def wrapper(*args, **kwargs): 
            content = func(*args, **kwargs) 
            print "<{tag}>{content}</{tag}>".format(tag=tag_name, content=content) 
        print &#39;end of inner wrapper function.&#39; 
        return wrapper 
    print &#39;end of outer function&#39; 
    return wrapper_ 
 
@html_tags(&#39;b&#39;) 
def hello(name=&#39;Toby&#39;): 
    return &#39;Hello {}!&#39;.format(name) 
 
hello() 
hello()

在装饰器中我在各个可能的位置都加上了print语句，用于记录被调用的情况。你知道他们最后打印出来的顺序吗?如果你心里没底，那么最好不要在装饰器函数之外添加逻辑功能，否则这个装饰器就不受你控制了。以下是输出结果：

begin outer function. 
end of outer function 
begin of inner wrapper function. 
end of inner wrapper function. 
<b>Hello Toby!</b> 
<b>Hello Toby!</b>

错误的函数签名和文档

装饰器装饰过的函数看上去名字没变，其实已经变了。

def logging(func): 
    def wrapper(*args, **kwargs): 
        """print log before a function.""" 
        print "[DEBUG] {}: enter {}()".format(datetime.now(), func.__name__) 
        return func(*args, **kwargs) 
    return wrapper 
 
@logging 
def say(something): 
    """say something""" 
    print "say {}!".format(something) 
 
print say.__name__  # wrapper

为什么会这样呢?只要你想想装饰器的语法糖@代替的东西就明白了。@等同于这样的写法。

say = logging(say)

logging其实返回的函数名字刚好是wrapper，那么上面的这个语句刚好就是把这个结果赋值给say，say的__name__自然也就是wrapper了，不仅仅是name，其他属性也都是来自wrapper，比如doc，source等等。

使用标准库里的functools.wraps，可以基本解决这个问题。

from functools import wraps 
 
def logging(func): 
    @wraps(func) 
    def wrapper(*args, **kwargs): 
        """print log before a function.""" 
        print "[DEBUG] {}: enter {}()".format(datetime.now(), func.__name__) 
        return func(*args, **kwargs) 
    return wrapper 
 
@logging 
def say(something): 
    """say something""" 
    print "say {}!".format(something) 
 
print say.__name__  # say 
print say.__doc__ # say something

看上去不错!主要问题解决了，但其实还不太完美。因为函数的签名和源码还是拿不到的。

import inspect 
print inspect.getargspec(say)  # failed 
print inspect.getsource(say)  # failed

如果要彻底解决这个问题可以借用第三方包，比如wrapt。后文有介绍。

不能装饰@staticmethod 或者 @classmethod

当你想把装饰器用在一个静态方法或者类方法时，不好意思，报错了。

class Car(object): 
    def __init__(self, model): 
        self.model = model 
 
    @logging  # 装饰实例方法，OK 
    def run(self): 
        print "{} is running!".format(self.model) 
 
    @logging  # 装饰静态方法，Failed 
    @staticmethod 
    def check_model_for(obj): 
        if isinstance(obj, Car): 
            print "The model of your car is {}".format(obj.model) 
        else: 
            print "{} is not a car!".format(obj) 
 
""" 
Traceback (most recent call last): 
... 
  File "example_4.py", line 10, in logging 
    @wraps(func) 
  File "C:\Python27\lib\functools.py", line 33, in update_wrapper 
    setattr(wrapper, attr, getattr(wrapped, attr)) 
AttributeError: 'staticmethod' object has no attribute '__module__' 
"""

前面已经解释了@staticmethod这个装饰器，其实它返回的并不是一个callable对象，而是一个staticmethod对象，那么它是不符合装饰器要求的(比如传入一个callable对象)，你自然不能在它之上再加别的装饰器。要解决这个问题很简单，只要把你的装饰器放在@staticmethod之前就好了，因为你的装饰器返回的还是一个正常的函数，然后再加上一个@staticmethod是不会出问题的。

class Car(object): 
    def __init__(self, model): 
        self.model = model 
 
    @staticmethod 
    @logging  # 在@staticmethod之前装饰，OK 
    def check_model_for(obj): 
        pass

如何优化你的装饰器

嵌套的装饰函数不太直观，我们可以使用第三方包类改进这样的情况，让装饰器函数可读性更好。

decorator.py

decorator.py 是一个非常简单的装饰器加强包。你可以很直观的先定义包装函数wrapper()，再使用decorate(func, wrapper)方法就可以完成一个装饰器。

from decorator import decorate 
 
def wrapper(func, *args, **kwargs): 
    """print log before a function.""" 
    print "[DEBUG] {}: enter {}()".format(datetime.now(), func.__name__) 
    return func(*args, **kwargs) 
 
def logging(func): 
    return decorate(func, wrapper)  # 用wrapper装饰func

你也可以使用它自带的@decorator装饰器来完成你的装饰器。

from decorator import decorator 
 
@decorator 
def logging(func, *args, **kwargs): 
    print "[DEBUG] {}: enter {}()".format(datetime.now(), func.__name__) 
    return func(*args, **kwargs)

decorator.py实现的装饰器能完整保留原函数的name，doc和args，唯一有问题的就是inspect.getsource(func)返回的还是装饰器的源代码，你需要改成inspect.getsource(func.__wrapped__)。

wrapt

wrapt是一个功能非常完善的包，用于实现各种你想到或者你没想到的装饰器。使用wrapt实现的装饰器你不需要担心之前inspect中遇到的所有问题，因为它都帮你处理了，甚至inspect.getsource(func)也准确无误。

import wrapt 
 
# without argument in decorator 
@wrapt.decorator 
def logging(wrapped, instance, args, kwargs):  # instance is must 
    print "[DEBUG]: enter {}()".format(wrapped.__name__) 
    return wrapped(*args, **kwargs) 
 
@logging 
def say(something): pass

使用wrapt你只需要定义一个装饰器函数，但是函数签名是固定的，必须是(wrapped, instance, args, kwargs)，注意第二个参数instance是必须的，就算你不用它。当装饰器装饰在不同位置时它将得到不同的值，比如装饰在类实例方法时你可以拿到这个类实例。根据instance的值你能够更加灵活的调整你的装饰器。另外，args和kwargs也是固定的，注意前面没有星号。在装饰器内部调用原函数时才带星号。

如果你需要使用wrapt写一个带参数的装饰器，可以这样写。

def logging(level): 
    @wrapt.decorator 
    def wrapper(wrapped, instance, args, kwargs): 
        print "[{}]: enter {}()".format(level, wrapped.__name__) 
        return wrapped(*args, **kwargs) 
    return wrapper 
 
@logging(level="INFO") 
def do(work): pass

关于wrapt的使用，建议查阅官方文档，在此不在赘述。

http://wrapt.readthedocs.io/e...

小结

Python’s decorators are not the same thing as Java’s annotations, nor are they the same as attributes in C#. They are two completely different concepts.

The concept of decorator is to strengthen the original function and object, which is equivalent to re-encapsulation, so generally the decorator function is named wrapper(), which means packaging. A function only performs its function when called. For example, the @logging decorator can output additional logs when the function is executed, and the function decorated with @cache can cache calculation results, etc.

Annotations and features add some attributes to the target function or object, which is equivalent to classifying it. These properties can be obtained through reflection, and different characteristic functions or objects can be intervened when the program is running. For example, functions with Setup are executed as preparation steps, or all functions with TestMethod are found and executed in sequence, etc.

So far I have finished talking about the decorators that I know about, but there are still some things that have not been mentioned, such as decorative decorators. Will add more when I have the chance. Thanks for watching.