Fully typed Python decorator with parameters

The code shown in this short article is taken from my small open source project designed by contract, which provides a typed decorator. Decorators are a very useful concept and you will definitely find a lot about them online. Simply put, they allow code to be executed every time (before and after) the decorated function is called. This way you can modify function parameters or return values, measure execution times, add logging, perform execution-time type checking, and more. Note that decorators can also be written for classes, providing another metaprogramming approach (such as is done in the attrs package)

In its simplest form, a decorator definition looks like the following code:

def my_first_decorator(func):
 def wrapped(*args, **kwargs):
 # do something before
 result = func(*args, **kwargs)
 # do something after
 return result
 return wrapped
@my_first_decorator
def func(a):
 return a

The above code, because when a wrapped nested function is defined, its surrounding variables can be accessed within the function and saved in memory, as long as the function is used somewhere (this called closures in functional programming languages).

Simple, but this has some disadvantages. The biggest problem is that the decorated function will lose its previous function name (you can see this with inspect.signature), its docstring, and even its name. These are problems with source code documentation tools (such as sphinx), But it can be easily solved using the functools.wraps decorator in the standard library:

from functools import wraps
from typing import Any, Callable, TypeVar, ParamSpec
P = ParamSpec("P") # 需要python >= 3.10
R = TypeVar("R")
def my_second_decorator(func: Callable[P, R]) -> Callable[P, R]:
 @wraps(func)
 def wrapped(*args: Any, **kwargs: Any) -> R:
 # do something before
 result = func(*args, **kwargs)
 # do something after
 return result
 return wrapped
@my_second_decorator
def func2(a: int) -> int:
 """Does nothing"""
 return a
print(func2.__name__)
# 'func2'
print(func2.__doc__)
# 'Does nothing'

In this example, I have added type annotations, and the annotations and type hints are done for Python The most important addition. Better readability, code completion in the IDE, and maintainability of larger code bases are just a few examples. The code above should already cover most use cases, but the decorator cannot be parameterized. Consider writing a decorator that records the execution time of a function, but only if it exceeds a certain number of seconds. This number should be individually configurable for each decorated function. If not specified, the default value should be used and the decorator should be used without parentheses for easier use:

@time(threshold=2)
def func1(a):
...
# No paranthesis when using default threshold
@time
def func2(b):
...

If you can use parentheses in the second case, Or don't provide default values ​​for the parameters at all, then this recipe will suffice:

from functools import wraps
from typing import Any, Callable, TypeVar, ParamSpec
P = ParamSpec("P") # 需要python >= 3.10
R = TypeVar("R")
def my_third_decorator(threshold: int = 1) -> Callable[[Callable[P, R]], Callable[P, R]]:
 def decorator(func: Callable[P, R]) -> Callable[P, R]:
 @wraps(func)
 def wrapper(*args: Any, **kwargs: Any) -> R:
 # do something before you can use `threshold`
 result = func(*args, **kwargs)
 # do something after
 return result
 return wrapper
 return decorator
@my_third_decorator(threshold=2)
def func3a(a: int) -> None:
...
# works
@my_third_decorator()
def func3b(a: int) -> None:
...
# Does not work!
@my_third_decorator
def func3c(a: int) -> None:
...

To cover the third case, there are packages, namely wraps and decorator, that can actually do more than just add available Select parameters. While the quality is very high, they introduce quite a bit of additional complexity. Using the wrapt-decorated function, I further encountered serialization issues when running the function on a remote cluster. As far as I know, neither is fully typed, so static type checkers/linters (such as mypy) fail in strict mode.

When I worked on my own package and decided to write my own solution, I had to solve these problems. It becomes a pattern that is easily reusable but difficult to convert into a library.

It uses the overloaded decorators of the standard library. This way, the same decorator can be specified to be used with our parameterless one. Other than that, it's a combination of the two snippets above. One drawback of this approach is that all parameters need to be given as keyword arguments (this increases readability after all)

from typing import Callable, TypeVar, ParamSpec
from functools import partial, wraps
P = ParamSpec("P") # requires python >= 3.10
R = TypeVar("R
@overload
def typed_decorator(func: Callable[P, R]) -> Callable[P, R]:
...
@overload
def typed_decorator(*, first: str = "x", second: bool = True) -> Callable[[Callable[P, R]], Callable[P, R]]:
...
def typed_decorator(
 func: Optional[Callable[P, R]] = None, *, first: str = "x", second: bool = True
) -> Union[Callable[[Callable[P, R]], Callable[P, R]], Callable[P, R]]:
 """
Describe what the decorator is supposed to do!
Parameters
----------
first : str, optional
First argument, by default "x".
This is a keyword-only argument!
second : bool, optional
Second argument, by default True.
This is a keyword-only argument!
"""
 def wrapper(func: Callable[P, R], *args: Any, **kw: Any) -> R:
 """The actual logic"""
 # Do something with first and second and produce a `result` of type `R`
 return result
 # Without arguments `func` is passed directly to the decorator
 if func is not None:
 if not callable(func):
 raise TypeError("Not a callable. Did you use a non-keyword argument?")
 return wraps(func)(partial(wrapper, func))
 # With arguments, we need to return a function that accepts the function
 def decorator(func: Callable[P, R]) -> Callable[P, R]:
 return wraps(func)(partial(wrapper, func))
 return decorator

Later, we can use ours separately without Decorators for parameters

@typed_decorator
def spam(a: int) -> int:
 return a
@typed_decorator(first = "y
def eggs(a: int) -> int:
 return a

This pattern definitely has some overhead, but the benefits outweigh the costs.

Original text:​​https://www.php.cn/link/d0f82e1046ccbd597c7f2a7bfba9e7dd​​

The above is the detailed content of Fully typed Python decorator with parameters. For more information, please follow other related articles on the PHP Chinese website!