Detailed usage introduction of Python decorators (code examples)

This article brings you a detailed usage introduction (code example) about Python decorators. It has certain reference value. Friends in need can refer to it. I hope it will be helpful to you.

In Python, decorators are generally used to decorate functions to implement public functions and achieve code reuse. Add @xxxx before the function definition, and then the function will inject certain behaviors, which is amazing! However, this is just syntactic sugar.

Scenario

Assume that there are some working functions used to process data differently:

def work_bar(data):
    pass


def work_foo(data):
    pass

We want to do this before/after the function call Output log, what should I do?

Fool’s solution

logging.info('begin call work_bar')
work_bar(1)
logging.info('call work_bar done')

What if there are multiple code calls? It scares me just thinking about it!

Function packaging

The fool's solution is nothing more than having too much code redundancy, and each function call must be written againlogging. This part of the redundant logic can be encapsulated into a new function:

def smart_work_bar(data):
    logging.info('begin call: work_bar')
    work_bar(data)
    logging.info('call doen: work_bar')

In this way, smart_work_bar can be called every time:

smart_work_bar(1)

# ...

smart_work_bar(some_data)

General closure

Looks perfect... However, when work_foo also has the same need, do we need to implement smart_work_foo again? This is obviously unscientific!

Don’t worry, we can use closures:

def log_call(func):
    def proxy(*args, **kwargs):
        logging.info('begin call: {name}'.format(name=func.func_name))
        result = func(*args, **kwargs)
        logging.info('call done: {name}'.format(name=func.func_name))
        return result
    return proxy

This function receives a function object (proxy function) as a parameter and returns a proxy function. When calling the proxy function, the log is output first, then the proxy function is called, the log is output after the call is completed, and finally the call result is returned. In this way, doesn’t it achieve the purpose of generalization? ——For any proxy function func, log_call can be easily handled.

smart_work_bar = log_call(work_bar)
smart_work_foo = log_call(work_foo)

smart_work_bar(1)
smart_work_foo(1)

# ...

smart_work_bar(some_data)
smart_work_foo(some_data)

In line 1, log_call receives the parameter work_bar, returns a proxy function proxy, and assigns it to smart_work_bar. In line 4, call smart_work_bar, which is the proxy function proxy, first output the log, and then call func, which is work_bar, and finally output the log. Notice that in the proxy function, func is closely related to the work_bar object passed in. This is the closure.

Again, you can overwrite the proxy function name. Prefixing a new name with smart_ is still a bit cumbersome:

work_bar = log_call(work_bar)
work_foo = log_call(work_foo)

work_bar(1)
work_foo(1)

Syntactic sugar

Let’s take a look at the following code first:

def work_bar(data):
    pass
work_bar = log_call(work_bar)


def work_foo(data):
    pass
work_foo = log_call(work_foo)

Although the code is not redundant, it is still not intuitive enough. At this time, syntax sugar comes~~~

@log_call
def work_bar(data):
    pass

So, pay attention to one thing (Emphasis ), the function of @log_call here is just: tell PythonCompiler inserts codework_bar = log_call(work_bar).

Evaluation Decorator

Let’s first guess what the decorator eval_now does?

def eval_now(func):
    return func()

It seems strange. There is no proxy function defined. Is it considered a decorator?

@eval_now
def foo():
    return 1

print foo

This code outputs 1, which is to call and evaluate the function. So what's the use? Can't we just write foo = 1 directly? In this simple example, it is of course possible to write like this. Let's look at a more complex example - initializing a log object:

# some other code before...

# log format
formatter = logging.Formatter(
    '[%(asctime)s] %(process)5d %(levelname) 8s - %(message)s',
    '%Y-%m-%d %H:%M:%S',
)

# stdout handler
stdout_handler = logging.StreamHandler(sys.stdout)
stdout_handler.setFormatter(formatter)
stdout_handler.setLevel(logging.DEBUG)

# stderr handler
stderr_handler = logging.StreamHandler(sys.stderr)
stderr_handler.setFormatter(formatter)
stderr_handler.setLevel(logging.ERROR)

# logger object
logger = logging.Logger(__name__)
logger.setLevel(logging.DEBUG)
logger.addHandler(stdout_handler)
logger.addHandler(stderr_handler)

# again some other code after...

Use eval_now:

# some other code before...

@eval_now
def logger():
    # log format
    formatter = logging.Formatter(
        '[%(asctime)s] %(process)5d %(levelname) 8s - %(message)s',
        '%Y-%m-%d %H:%M:%S',
    )

    # stdout handler
    stdout_handler = logging.StreamHandler(sys.stdout)
    stdout_handler.setFormatter(formatter)
    stdout_handler.setLevel(logging.DEBUG)

    # stderr handler
    stderr_handler = logging.StreamHandler(sys.stderr)
    stderr_handler.setFormatter(formatter)
    stderr_handler.setLevel(logging.ERROR)

    # logger object
    logger = logging.Logger(__name__)
    logger.setLevel(logging.DEBUG)
    logger.addHandler(stdout_handler)
    logger.addHandler(stderr_handler)

    return logger

# again some other code after...

The purpose of the two pieces of code is the same, But the latter is obviously clearer and has the style of code blocks. More importantly, function calls are initialized in the local namespace to avoid temporary variables (such as formatter, etc.) from polluting external namespaces (such as the global).

Decorator with parameters

Define a decorator for recording slow function calls:

def log_slow_call(func):
    def proxy(*args, **kwargs):
        start_ts = time.time()
        result = func(*args, **kwargs)
        end_ts = time.time()

        seconds = start_ts - end_ts
        if seconds > 1:
        logging.warn('slow call: {name} in {seconds}s'.format(
            name=func.func_name,
            seconds=seconds,
        ))

        return result

    return proxy

3、5## The # line samples the current time before and after the function call, and the 7 line calculates the call time. If it takes more than one second, a warning log is output.

@log_slow_call
def sleep_seconds(seconds):
    time.sleep(seconds)

sleep_seconds(0.1)  # 没有日志输出

sleep_seconds(2)    # 输出警告日志

However, the threshold setting always depends on the situation, and different functions may set different values. It would be nice if there was a way to parameterize the threshold:

def log_slow_call(func, threshold=1):
    def proxy(*args, **kwargs):
        start_ts = time.time()
        result = func(*args, **kwargs)
        end_ts = time.time()

        seconds = start_ts - end_ts
        if seconds > threshold:
        logging.warn('slow call: {name} in {seconds}s'.format(
            name=func.func_name,
            seconds=seconds,
        ))

        return result

    return proxy

However,

@xxxxThe syntactic sugar always calls the decorator with the decorated function as the parameter, which means there is no chance to pass thresholdparameter. How to do it? ——Use a closure to encapsulate the threshold parameter:

def log_slow_call(threshold=1):
    def decorator(func):
        def proxy(*args, **kwargs):
            start_ts = time.time()
            result = func(*args, **kwargs)
            end_ts = time.time()

            seconds = start_ts - end_ts
            if seconds > threshold:
            logging.warn('slow call: {name} in {seconds}s'.format(
                name=func.func_name,
                seconds=seconds,
            ))

            return result

        return proxy

    return decorator


@log_slow_call(threshold=0.5)
def sleep_seconds(seconds):
    time.sleep(seconds)

In this way,

log_slow_call(threshold=0.5) calls the return function decorator, and the function has Closure variable threshold, value is 0.5. decoratorRedecorationsleep_seconds.

Using the default threshold, the function call cannot be omitted:

@log_slow_call()
def sleep_seconds(seconds):
    time.sleep(seconds)

Virgos may be unhappy with the pair of brackets in the first line, so you can improve it like this:

def log_slow_call(func=None, threshold=1):
    def decorator(func):
        def proxy(*args, **kwargs):
            start_ts = time.time()
            result = func(*args, **kwargs)
            end_ts = time.time()

            seconds = start_ts - end_ts
            if seconds > threshold:
            logging.warn('slow call: {name} in {seconds}s'.format(
                name=func.func_name,
                seconds=seconds,
            ))

            return result

        return proxy

    if func is None:
        return decorator
    else:
        return decorator(func)

This This writing method is compatible with two different usages, usage

A default threshold (no call); usage B custom threshold (with call).

# Case A
@log_slow_call
def sleep_seconds(seconds):
    time.sleep(seconds)


# Case B
@log_slow_call(threshold=0.5)
def sleep_seconds(seconds):
    time.sleep(seconds)

Usage

A, what happens is log_slow_call(sleep_seconds), that is, the func parameter is non-empty, this is a direct adjustmentdecoratorWrap and return (the threshold is the default).

用法B中，先发生的是log_slow_call(threshold=0.5)，func参数为空，直接返回新的装饰器decorator，关联闭包变量threshold，值为0.5；然后，decorator再装饰函数sleep_seconds，即decorator(sleep_seconds)。注意到，此时threshold关联的值是0.5，完成定制化。

你可能注意到了，这里最好使用关键字参数这种调用方式——使用位置参数会很丑陋：

# Case B-
@log_slow_call(None, 0.5)
def sleep_seconds(seconds):
    time.sleep(seconds)

当然了，函数调用尽量使用关键字参数是一种极佳实践，含义清晰，在参数很多的情况下更是如此。

智能装饰器

上节介绍的写法，嵌套层次较多，如果每个类似的装饰器都用这种方法实现，还是比较费劲的(脑子不够用)，也比较容易出错。

假设有一个智能装饰器smart_decorator，修饰装饰器log_slow_call，便可获得同样的能力。这样，log_slow_call定义将变得更清晰，实现起来也更省力啦：

@smart_decorator
def log_slow_call(func, threshold=1):
    def proxy(*args, **kwargs):
        start_ts = time.time()
        result = func(*args, **kwargs)
        end_ts = time.time()

        seconds = start_ts - end_ts
        if seconds > threshold:
        logging.warn('slow call: {name} in {seconds}s'.format(
            name=func.func_name,
            seconds=seconds,
        ))

        return result

    return proxy

脑洞开完，smart_decorator如何实现呢？其实也简单：

def smart_decorator(decorator):

    def decorator_proxy(func=None, **kwargs):
        if func is not None:
            return decorator(func=func, **kwargs)

        def decorator_proxy(func):
            return decorator(func=func, **kwargs)

        return decorator_proxy

    return decorator_proxy

smart_decorator实现了以后，设想就成立了！这时，log_slow_call，就是decorator_proxy(外层)，关联的闭包变量decorator是本节最开始定义的log_slow_call(为了避免歧义，称为real_log_slow_call)。log_slow_call支持以下各种用法：

# Case A
@log_slow_call
def sleep_seconds(seconds):
    time.sleep(seconds)

用法A中，执行的是decorator_proxy(sleep_seconds)(外层)，func非空，kwargs为空；直接执行decorator(func=func, **kwargs)，即real_log_slow_call(sleep_seconds)，结果是关联默认参数的proxy。

# Case B
# Same to Case A
@log_slow_call()
def sleep_seconds(seconds):
    time.sleep(seconds)

用法B中，先执行decorator_proxy()，func及kwargs均为空，返回decorator_proxy对象(内层)；再执行decorator_proxy(sleep_seconds)(内层)；最后执行decorator(func, **kwargs)，等价于real_log_slow_call(sleep_seconds)，效果与用法A一致。

# Case C
@log_slow_call(threshold=0.5)
def sleep_seconds(seconds):
    time.sleep(seconds)

用法C中，先执行decorator_proxy(threshold=0.5)，func为空但kwargs非空，返回decorator_proxy对象(内层)；再执行decorator_proxy(sleep_seconds)(内层)；最后执行decorator(sleep_seconds, **kwargs)，等价于real_log_slow_call(sleep_seconds, threshold=0.5)，阈值实现自定义！

The above is the detailed content of Detailed usage introduction of Python decorators (code examples). For more information, please follow other related articles on the PHP Chinese website!