search
HomeBackend DevelopmentPython TutorialLet's dive into the internals of Python classes

python video tutorialThe column introduces the internals of Python classes.

Let's dive into the internals of Python classes

This article will talk with you about some concepts and implementation principles behind classes and objects in Python 3.8. It mainly tries to explain the properties of Python classes and objects. Optimization support for storage, functions and methods, descriptors, object memory usage, and related issues such as inheritance and attribute lookup.

Let's start with a simple example:

class Employee:

    outsource = False

    def __init__(self, department, name):
        self.department = department
        self.name = name    @property
    def inservice(self):
        return self.department is not None

    def __repr__(self):
        return f"<employee:>"employee = Employee('IT', 'bobo')复制代码</employee:>

employee The object is an instance of the Employee class and it has two propertiesdepartment and name, whose values ​​belong to this instance. outsource is a class attribute, the owner is the class, and all instance objects of the class share this attribute value, which is consistent with other object-oriented languages.

Changing a class variable will affect all instance objects of the class:

>>> e1 = Employee('IT', 'bobo')>>> e2 = Employee('HR', 'cici')>>> e1.outsource, e2.outsource
(False, False)>>> Employee.outsource = True>>> e1.outsource, e2.outsource>>> (True, True)复制代码

This is limited to changes from the class, when we change the class variable from the instance:

>>> e1 = Employee('IT', 'bobo')>>> e2 = Employee('HR', 'cici')>>> e1.outsource, e2.outsource
(False, False)>>> e1.outsource = True>>> e1.outsource, e2.outsource
(True, False)复制代码

Yes Yes, when you try to modify a class variable from an instance object, Python does not change the class variable value of the class, but creates an instance property with the same name, which is very correct and safe. Instance variables are given priority over class variables when searching for property values, as explained in detail in the Inheritance and Property Lookup section.

It is important to note that when the type of class variables is mutable type, you change them from the instance object:

>>> class S:...     L = [1, 2]
...>>> s1, s2 = S(), S()>>> s1.L, s2.L
([1, 2], [1, 2])>>> t1.L.append(3)>>> t1.L, s2.L
([1, 2, 3], [1, 2, 3])复制代码

Good practice The method is to avoid such a design as much as possible.

Storage of attributes

In this section, let’s take a look at how class attributes, methods and instance attributes in Python are associated and stored.

Instance attributes

In Python, all instance attributes are stored in the __dict__ dictionary, which is a regular dict. For instance attributes The maintenance is to obtain and modify from this dictionary, which is completely open to developers.

>>> e = Employee('IT', 'bobo')>>> e.__dict__
{'department': 'IT', 'name': 'bobo'}>>> type(e.__dict__)dict>>> e.name is e.__dict__['name']True>>> e.__dict__['department'] = 'HR'>>> e.department'HR'复制代码

Because instance attributes are stored in a dictionary, we can easily add or delete fields to the object at any time:

>>> e.age = 30 # 并没有定义 age 属性>>> e.age30>>> e.__dict__
{'department': 'IT', 'name': 'bobo', 'age': 30}>>> del e.age>>> e.__dict__
{'department': 'IT', 'name': 'd'}复制代码

We can also instantiate an object from the dictionary, Or restore the instance by saving the instance's __dict__.

>>> def new_employee_from(d):...     instance = object.__new__(Employee)...     instance.__dict__.update(d)...     return instance
...>>> e1 = new_employee_from({'department': 'IT', 'name': 'bobo'})>>> e1
<employee:>>>> state = e1.__dict__.copy()>>> del e1>>> e2 = new_employee_from(state)>>> e2>>> <employee:>复制代码</employee:></employee:>

Because __dict__ is completely open, we can add any hashable immutable key to it, such as numbers:

>>> e.__dict__[1] = 1>>> e.__dict__
{'department': 'IT', 'name': 'bobo', 1: 1}复制代码

These non-string fields cannot be accessed through instance objects. In order to ensure that such a situation does not occur, it is generally best not to write directly to __dict__ unless necessary. Don't even manipulate __dict__ directly.

So there is a saying that Python is a "consenting adults language".

This dynamic implementation makes our code very flexible and very convenient in many cases, but it also comes with storage and performance overhead. Therefore, Python also provides another mechanism (__slots__) to abandon the use of __dict__ to save memory and improve performance. See the __slots__ section for details.

Class attributes

Similarly, class attributes are also stored in the __dict__ dictionary of the class:

>>> Employee.__dict__
mappingproxy({'__module__': '__main__',              'outsource': True,              '__init__': <function>,              'inservice': <property>,              '__repr__': <function>,              '__str__': <function>,              '__dict__': <attribute>,              '__weakref__': <attribute>,              '__doc__': None}>>> type(Employee.__dict__)
mappingproxy复制代码</attribute></attribute></function></function></property></function>

is different from the "open" of the instance dictionary , the dictionary used by the class attribute is a MappingProxyType object, which is a dictionary that cannot be setattr. This means that it is read-only for developers, and its purpose is to ensure that the keys of class attributes are all strings to simplify and speed up the search logic of new class attributes and __mro__.

>>> Employee.__dict__['outsource'] = FalseTypeError: 'mappingproxy' object does not support item assignment复制代码

Because all methods belong to a class, they are also stored in the class dictionary. From the above example, you can see the existing __init__ and __repr__ method. We can add a few more to verify:

class Employee:
    # ...    @staticmethod
    def soo():
        pass    @classmethod
    def coo(cls):
        pass

    def foo(self):
        pass复制代码
>>> Employee.__dict__
mappingproxy({'__module__': '__main__',              'outsource': False,              '__init__': <function>,              '__repr__': <function>,              'inservice': <property>,              'soo': <staticmethod>,              'coo': <classmethod>,              'foo': <function>,              '__dict__': <attribute>,              '__weakref__': <attribute>,              '__doc__': None})复制代码</attribute></attribute></function></classmethod></staticmethod></property></function></function>

Inheritance and attribute lookup

So far, we already know that all attributes and methods are stored in two __dict__ In the dictionary, now let's take a look at how Python performs attribute lookup.

In Python 3, all classes implicitly inherit from object, so there is always an inheritance relationship, and Python supports multiple inheritance:

>>> class A:...     pass...>>> class B:...     pass...>>> class C(B):...     pass...>>> class D(A, C):...     pass...>>> D.mro()
[<class>, <class>, <class>, <class>, <class>]复制代码</class></class></class></class></class>

mro() is a special method that returns the linear parsing order of the class.

The default behavior of attribute access is to get, set or delete attributes from the object's dictionary. For example, the simple description of the lookup for e.f is:

e.f 的查找顺序会从 e.__dict__['f'] 开始,然后是 type(e).__dict__['f'],接下来依次查找 type(e) 的基类(__mro__ 顺序,不包括元类)。 如果找到的值是定义了某个描述器方法的对象,则 Python 可能会重载默认行为并转而发起调用描述器方法。这具体发生在优先级链的哪个环节则要根据所定义的描述器方法及其被调用的方式来决定。

所以,要理解查找的顺序,你必须要先了解描述器协议。

简单总结,有两种描述器类型:数据描述器和和非数据描述器。

如果一个对象除了定义 __get__() 之外还定义了 __set__()__delete__(),则它会被视为数据描述器。仅定义了 __get__() 的描述器称为非数据描述器(它们通常被用于方法,但也可以有其他用途)

由于函数只实现 __get__,所以它们是非数据描述器。

Python 的对象属性查找顺序如下:

  1. 类和父类字典的数据描述器
  2. 实例字典
  3. 类和父类字典中的非数据描述器

请记住,无论你的类有多少个继承级别,该类对象的实例字典总是存储了所有的实例变量,这也是 super 的意义之一。

下面我们尝试用伪代码来描述查找顺序:

def get_attribute(obj, name):
    class_definition = obj.__class__

    descriptor = None
    for cls in class_definition.mro():        if name in cls.__dict__:
            descriptor = cls.__dict__[name]            break

    if hasattr(descriptor, '__set__'):        return descriptor, 'data descriptor'

    if name in obj.__dict__:        return obj.__dict__[name], 'instance attribute'

    if descriptor is not None:        return descriptor, 'non-data descriptor'
    else:        raise AttributeError复制代码
>>> e = Employee('IT', 'bobo')>>> get_attribute(e, 'outsource')
(False, 'non-data descriptor')>>> e.outsource = True>>> get_attribute(e, 'outsource')
(True, 'instance attribute')>>> get_attribute(e, 'name')
('bobo', 'instance attribute')>>> get_attribute(e, 'inservice')
(<property>, 'data descriptor')>>> get_attribute(e, 'foo')
(<function>, 'non-data descriptor')复制代码</function></property>

由于这样的优先级顺序,所以实例是不能重载类的数据描述器属性的,比如 property 属性:

>>> class Manager(Employee):...     def __init__(self, *arg):...         self.inservice = True...         super().__init__(*arg)
...>>> m = Manager("HR", "cici")
AttributeError: can't set attribute复制代码

发起描述器调用

上面讲到,在查找属性时,如果找到的值是定义了某个描述器方法的对象,则 Python 可能会重载默认行为并转而发起描述器方法调用。

描述器的作用就是绑定对象属性,我们假设 a 是一个实现了描述器协议的对象,对 e.a 发起描述器调用有以下几种情况:

  • 直接调用:用户级的代码直接调用e.__get__(a),不常用
  • 实例绑定:绑定到一个实例,e.a 会被转换为调用: type(e).__dict__['a'].__get__(e, type(e))
  • 类绑定:绑定到一个类,E.a 会被转换为调用: E.__dict__['a'].__get__(None, E)

在继承关系中进行绑定时,会根据以上情况和 __mro__ 顺序来发起链式调用。

函数与方法

我们知道方法是属于特定类的函数,唯一的不同(如果可以算是不同的话)是方法的第一个参数往往是为类或实例对象保留的,在 Python 中,我们约定为 clsself, 当然你也可以取任何名字如 this(只是最好不要这样做)。

上一节我们知道,函数实现了 __get__() 方法的对象,所以它们是非数据描述器。在 Python 访问(调用)方法支持中正是通过调用 __get__() 将调用的函数绑定成方法的。

在纯 Python 中,它的工作方式如下(示例来自描述器使用指南):

class Function:
    def __get__(self, obj, objtype=None):
        if obj is None:            return self        return types.MethodType(self, obj) # 将函数绑定为方法复制代码

在 Python 2 中,有两种方法: unbound method 和 bound method,在 Python 3 中只有后者。

bound method 与它们绑定的类或实例数据相关联:

>>> Employee.coo
<bound>>
>>> Employee.foo<function>
>>> e = Employee('IT', 'bobo')
>>> e.foo<bound>>复制代码</bound></function></bound>

我们可以从方法来访问实例与类:

>>> e.foo.__self__
<employee:>>>> e.foo.__self__.__class__
__main__.Employee复制代码</employee:>

借助描述符协议,我们可以在类的外部作用域手动绑定一个函数到方法,以访问类或实例中的数据,我将以这个示例来解释当你的对象访问(调用)类字典中存储的函数时将其绑定成方法(执行)的过程

现有以下函数:

>>> def f1(self):...     if isinstance(self, type):...         return self.outsource...     return self.name
...>>> bound_f1 = f1.__get__(e, Employee) # or bound_f1 = f1.__get__(e)>>> bound_f1
<bound>>>>> bound_f1.__self__
<employee:>>>> bound_f1()'bobo'复制代码</employee:></bound>

总结一下:当我们调用 e.foo() 时,首先从 Employee.__dict__['foo'] 中得到 foo 函数,在调用该函数的 foo 方法 foo.__get__(e) 将其转换成方法,然后执行 foo() 获得结果。这就完成了 e.foo() -> f(e) 的过程。

如果你对我的解释感到疑惑,我建议你可以阅读官方的描述器使用指南以进一步了解描述器协议,在该文的函数和方法和静态方法和类方法一节中详细了解函数绑定为方法的过程。同时在 Python 类一文的方法对象一节中也有相关的解释。

__slots__

Python 的对象属性值都是采用字典存储的,当我们处理数成千上万甚至更多的实例时,内存消耗可能是一个问题,因为字典哈希表的实现,总是为每个实例创建了大量的内存。所以 Python 提供了一种 __slots__ 的方式来禁用实例使用 __dict__,以优化此问题。

通过 __slots__ 来指定属性后,会将属性的存储从实例的 __dict__ 改为类的 __dict__ 中:

class Test:
    __slots__ = ('a', 'b')    def __init__(self, a, b):
        self.a = a
        self.b = b复制代码
>>> t = Test(1, 2)>>> t.__dict__
AttributeError: 'Test' object has no attribute '__dict__'>>> Test.__dict__
mappingproxy({'__module__': '__main__',              '__slots__': ('a', 'b'),              '__init__': <function>,              'a': <member>,              'b': <member>,              '__doc__': None})复制代码</member></member></function>

关于 __slots__ 我之前专门写过一篇文章分享过,感兴趣的同学请移步理解 Python 类属性 __slots__ 一文。

补充

__getattribute__ 和 __getattr__

也许你还有疑问,那函数的 __get__ 方法是怎么被调用的呢,这中间过程是什么样的?

在 Python 中 一切皆对象,所有对象都有一个默认的方法 __getattribute__(self, name)

该方法会在我们使用 . 访问 obj 的属性时会自动调用,为了防止递归调用,它总是实现为从基类 object 中获取 object.__getattribute__(self, name), 该方法大部分情况下会默认从 self__dict__ 字典中查找 name(除了特殊方法的查找)。

话外:如果该类还实现了 __getattr__则只有 __getattribute__ 显式地调用或是引发了 AttributeError 异常后才会被调用__getattr__ 由开发者自己实现,应当返回属性值或引发 AttributeError 异常。

而描述器正是由 __getattribute__() 方法调用,其大致逻辑为:

def __getattribute__(self, key):
    v = object.__getattribute__(self, key)    if hasattr(v, '__get__'):        return v.__get__(self)    return v复制代码

请注意:重写 __getattribute__() 会阻止描述器的自动调用。

函数属性

函数也是 Python function 对象,所以一样,它也具有任意属性,这有时候是有用的,比如实现一个简单的函数调用跟踪装饰器:

def calltracker(func):    @wraps(func)
    def wrapper(*args, **kwargs):
        wrapper.calls += 1
        return func(*args, **kwargs)
    wrapper.calls = 0
    return wrapper@calltrackerdef f():
    return 'f called'复制代码
>>> f.calls0>>> f()'f called'>>> f.calls1复制代码

相关免费学习推荐:python视频教程

The above is the detailed content of Let's dive into the internals of Python classes. For more information, please follow other related articles on the PHP Chinese website!

Statement
This article is reproduced at:juejin. If there is any infringement, please contact admin@php.cn delete
The Main Purpose of Python: Flexibility and Ease of UseThe Main Purpose of Python: Flexibility and Ease of UseApr 17, 2025 am 12:14 AM

Python's flexibility is reflected in multi-paradigm support and dynamic type systems, while ease of use comes from a simple syntax and rich standard library. 1. Flexibility: Supports object-oriented, functional and procedural programming, and dynamic type systems improve development efficiency. 2. Ease of use: The grammar is close to natural language, the standard library covers a wide range of functions, and simplifies the development process.

Python: The Power of Versatile ProgrammingPython: The Power of Versatile ProgrammingApr 17, 2025 am 12:09 AM

Python is highly favored for its simplicity and power, suitable for all needs from beginners to advanced developers. Its versatility is reflected in: 1) Easy to learn and use, simple syntax; 2) Rich libraries and frameworks, such as NumPy, Pandas, etc.; 3) Cross-platform support, which can be run on a variety of operating systems; 4) Suitable for scripting and automation tasks to improve work efficiency.

Learning Python in 2 Hours a Day: A Practical GuideLearning Python in 2 Hours a Day: A Practical GuideApr 17, 2025 am 12:05 AM

Yes, learn Python in two hours a day. 1. Develop a reasonable study plan, 2. Select the right learning resources, 3. Consolidate the knowledge learned through practice. These steps can help you master Python in a short time.

Python vs. C  : Pros and Cons for DevelopersPython vs. C : Pros and Cons for DevelopersApr 17, 2025 am 12:04 AM

Python is suitable for rapid development and data processing, while C is suitable for high performance and underlying control. 1) Python is easy to use, with concise syntax, and is suitable for data science and web development. 2) C has high performance and accurate control, and is often used in gaming and system programming.

Python: Time Commitment and Learning PacePython: Time Commitment and Learning PaceApr 17, 2025 am 12:03 AM

The time required to learn Python varies from person to person, mainly influenced by previous programming experience, learning motivation, learning resources and methods, and learning rhythm. Set realistic learning goals and learn best through practical projects.

Python: Automation, Scripting, and Task ManagementPython: Automation, Scripting, and Task ManagementApr 16, 2025 am 12:14 AM

Python excels in automation, scripting, and task management. 1) Automation: File backup is realized through standard libraries such as os and shutil. 2) Script writing: Use the psutil library to monitor system resources. 3) Task management: Use the schedule library to schedule tasks. Python's ease of use and rich library support makes it the preferred tool in these areas.

Python and Time: Making the Most of Your Study TimePython and Time: Making the Most of Your Study TimeApr 14, 2025 am 12:02 AM

To maximize the efficiency of learning Python in a limited time, you can use Python's datetime, time, and schedule modules. 1. The datetime module is used to record and plan learning time. 2. The time module helps to set study and rest time. 3. The schedule module automatically arranges weekly learning tasks.

Python: Games, GUIs, and MorePython: Games, GUIs, and MoreApr 13, 2025 am 12:14 AM

Python excels in gaming and GUI development. 1) Game development uses Pygame, providing drawing, audio and other functions, which are suitable for creating 2D games. 2) GUI development can choose Tkinter or PyQt. Tkinter is simple and easy to use, PyQt has rich functions and is suitable for professional development.

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
1 months agoBy尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. Best Graphic Settings
1 months agoBy尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. How to Fix Audio if You Can't Hear Anyone
1 months agoBy尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. Chat Commands and How to Use Them
1 months agoBy尊渡假赌尊渡假赌尊渡假赌

Hot Tools

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

SublimeText3 English version

SublimeText3 English version

Recommended: Win version, supports code prompts!

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

MantisBT

MantisBT

Mantis is an easy-to-deploy web-based defect tracking tool designed to aid in product defect tracking. It requires PHP, MySQL and a web server. Check out our demo and hosting services.

VSCode Windows 64-bit Download

VSCode Windows 64-bit Download

A free and powerful IDE editor launched by Microsoft