Magic methods in Python

Magic methods in python are special methods that allow you to add "magic" to a class. They are often named surrounded by two underscores.

Python’s magic method, also known as the dunder (double underline) method. Most of the time, we use them for simple things like constructors (init), string representations (str, repr) or arithmetic operators (add/mul). In fact, there are many methods that you may not have heard of but are very useful. In this article, we will sort out these magic methods!

Iterator size

We all know the __len__ method, which can be used to implement the len() function on the container class. But what if you want to get the length of a class object that implements an iterator?

it = iter(range(100))
 print(it.__length_hint__())
 # 100
 next(it)
 print(it.__length_hint__())
 # 99
 
 a = [1, 2, 3, 4, 5]
 it = iter(a)
 print(it.__length_hint__())
 # 5
 next(it)
 print(it.__length_hint__())
 # 4
 a.append(6)
 print(it.__length_hint__())
 # 5

All you need to do is implement the __length_hint__ method, which is a built-in method on iterators (not generators) , as you can see above, and also supports dynamic length changes. However, true to its name, this is just a hint and is not guaranteed to be completely accurate: for list iterators, you can get accurate results, but for other iterators it is not sure. But even if it's not accurate, it can help us get the information we need, as explained in PEP 424.

length_hint must return an integer (else a TypeError is raised) or NotImplemented, and is not required to be accurate. It may return a value that is either larger or smaller than the actual size of the container. A return value of NotImplemented indicates that there is no finite length estimate. It may not return a negative value (else a ValueError is raised).

Metaprogramming

Most of the rarely seen magical methods are related to metaprogramming. Although metaprogramming may not be something we need to use every day, there are some handy tricks that can use it.

One such trick is to use __init_subclass__ as a shortcut to extend the functionality of a base class without having to deal with metaclasses:

 class Pet:
def __init_subclass__(cls, /, default_breed, **kwargs):
super().__init_subclass__(**kwargs)
cls.default_breed = default_breed
 
 class Dog(Pet, default_name="German Shepherd"):
pass

In the code above we add keyword arguments to the base class, the Parameters can be set when defining the subclass. A real use case might be to use this method when you want to handle the supplied parameters rather than just assigning to a property.

It seems very obscure and rarely used, but in fact you may have encountered it many times, because it is generally used when building APIs, such as in SQLAlchemy or Flask Views. arrive.

Another metaclass magic method is __call__. This method allows customization of what happens when a class instance is called:

 class CallableClass:
def __call__(self, *args, **kwargs):
print("I was called!")
 
 instance = CallableClass()
 
 instance()
 # I was called!

You can use this to create a class that cannot be called:

 class NoInstances(type):
def __call__(cls, *args, **kwargs):
raise TypeError("Can't create instance of this class")
 
 class SomeClass(metaclass=NoInstances):
@staticmethod
def func(x):
print('A static method')
 
 instance = SomeClass()
 # TypeError: Can't create instance of this class

For classes with only static methods, there is no need to create the class This method is used in the example.

Another similar scenario is the singleton pattern - a class can only have at most one instance:

 class Singleton(type):
def __init__(cls, *args, **kwargs):
cls.__instance = None
super().__init__(*args, **kwargs)
 
def __call__(cls, *args, **kwargs):
if cls.__instance is None:
cls.__instance = super().__call__(*args, **kwargs)
return cls.__instance
else:
return cls.__instance
 
 class Logger(metaclass=Singleton):
def __init__(self):
print("Creating global Logger instance")

Singleton class has a private __instance - if not, it will be created and Assigned, it will only be returned if it already exists.

Suppose there is a class and you want to create an instance of it without calling __init__. The __new__ method can help solve this problem:

 class Document:
def __init__(self, text):
self.text = text
 
 bare_document = Document.__new__(Document)
 print(bare_document.text)
 # AttributeError: 'Document' object has no attribute 'text'
 
 setattr(bare_document, "text", "Text of the document")

In some cases, we may need to bypass the usual process of creating an instance, and the above code demonstrates how to do this. Instead of calling Document(…), we call Document.__new__(Document), which creates a bare instance without calling __init__. Therefore, the attribute of the instance (text in this case) is not initialized, so we need to additionally use the setattr function to assign the value (it is also a magic method __setattr__).

Why do you do that. Because we may want to replace the constructor, such as:

 class Document:
def __init__(self, text):
self.text = text
 
@classmethod
def from_file(cls, file): # Alternative constructor
d = cls.__new__(cls)
# Do stuff...
return d

The from_file method is defined here, which acts as a constructor, first using __new__ to create an instance, and then configuring it without calling __init__.

The next magical method related to metaprogramming is __getattr__. This method is called when normal property access fails. This can be used to delegate access/calls to missing methods to another class:

 class String:
def __init__(self, value):
self._value = str(value)
 
def custom_operation(self):
pass
 
def __getattr__(self, name):
return getattr(self._value, name)
 
 s = String("some text")
 s.custom_operation() # Calls String.custom_operation()
 print(s.split()) # Calls String.__getattr__("split") and delegates to str.split
 # ['some', 'text']
 
 print("some text" + "more text")
 # ... works
 print(s + "more text")
 # TypeError: unsupported operand type(s) for +: 'String' and 'str'

We want to add some extra functions to the class (like custom_operation above) to define a custom implementation of string. But we don't want to reimplement every string method, such as split, join, capitalize, etc. Here we can use __getattr__ to call these existing string methods.

While this works for normal methods, please note that in the example above, the magic method __add__ (which provides operations such as connections) does not get a delegate. So, if we want them to work as well, we have to reimplement them.

自省(introspection)

最后一个与元编程相关的方法是__getattribute__。它一个看起来非常类似于前面的__getattr__，但是他们有一个细微的区别，__getattr__只在属性查找失败时被调用，而__getattribute__是在尝试属性查找之前被调用。

所以可以使用__getattribute__来控制对属性的访问，或者你可以创建一个装饰器来记录每次访问实例属性的尝试:

 def logger(cls):
original_getattribute = cls.__getattribute__
 
def getattribute(self, name):
print(f"Getting: '{name}'")
return original_getattribute(self, name)
 
cls.__getattribute__ = getattribute
return cls
 
 @logger
 class SomeClass:
def __init__(self, attr):
self.attr = attr
 
def func(self):
...
 
 instance = SomeClass("value")
 instance.attr
 # Getting: 'attr'
 instance.func()
 # Getting: 'func'

装饰器函数logger 首先记录它所装饰的类的原始__getattribute__方法。然后将其替换为自定义方法，该方法在调用原始的__getattribute__方法之前记录了被访问属性的名称。

魔法属性

到目前为止，我们只讨论了魔法方法，但在Python中也有相当多的魔法变量/属性。其中一个是__all__:

 # some_module/__init__.py
 __all__ = ["func", "some_var"]
 
 some_var = "data"
 some_other_var = "more data"
 
 def func():
return "hello"
 
 # -----------
 
 from some_module import *
 
 print(some_var)
 # "data"
 print(func())
 # "hello"
 
 print(some_other_var)
 # Exception, "some_other_var" is not exported by the module

这个属性可用于定义从模块导出哪些变量和函数。我们创建了一个Python模块…/some_module/单独文件(__init__.py)。在这个文件中定义了2个变量和一个函数，只导出其中的2个(func和some_var)。如果我们尝试在其他Python程序中导入some_module的内容，我们只能得到2个内容。

但是要注意，__all__变量只影响上面所示的* import，我们仍然可以使用显式的名称导入函数和变量，比如import some_other_var from some_module。

另一个常见的双下划线变量(模块属性)是__file__。这个变量标识了访问它的文件的路径:

 from pathlib import Path
 
 print(__file__)
 print(Path(__file__).resolve())
 # /home/.../directory/examples.py
 
 # Or the old way:
 import os
 print(os.path.dirname(os.path.abspath(__file__)))
 # /home/.../directory/

这样我们就可以结合__all__和__file__，可以在一个文件夹中加载所有模块:

# Directory structure:
 # .
 # |____some_dir
 # |____module_three.py
 # |____module_two.py
 # |____module_one.py
 
 from pathlib import Path, PurePath
 modules = list(Path(__file__).parent.glob("*.py"))
 print([PurePath(f).stem for f in modules if f.is_file() and not f.name == "__init__.py"])
 # ['module_one', 'module_two', 'module_three']

最后一个我重要的属性是的是__debug__。它可以用于调试，但更具体地说，它可以用于更好地控制断言:

 # example.py
 def func():
if __debug__:
print("debugging logs")
 
# Do stuff...
 
 func()

如果我们使用python example.py正常运行这段代码，我们将看到打印出“调试日志”，但是如果我们使用python -O example.py，优化标志(-O)将把__debug__设置为false并删除调试消息。因此，如果在生产环境中使用-O运行代码，就不必担心调试过程中被遗忘的打印调用，因为它们都不会显示。

创建自己魔法方法？

我们可以创建自己的方法和属性吗?是的，你可以，但你不应该这么做。

双下划线名称是为Python语言的未来扩展保留的，不应该用于自己的代码。如果你决定在你的代码中使用这样的名称，那么将来如果它们被添加到Python解释器中，这就与你的代码不兼容了。所以对于这些方法，我们只要记住和使用就好了。

The above is the detailed content of Magic methods in Python. For more information, please follow other related articles on the PHP Chinese website!