Python pitfalls and precautions often encountered during development

I have encountered some pitfalls when using Python recently, such as using the variable object datetime.datetime.now() as the default parameter of the function, module circular dependencies, etc.

Record it here for future inquiry and addition.

Avoid mutable objects as default parameters

In the process of using functions, default parameters are often involved. In Python, when using mutable objects as default parameters, unexpected results may occur.

Let’s look at an example below:

def append_item(a = 1, b = []):
    b.append(a)
    print b
     
append_item(a=1)
append_item(a=3)
append_item(a=5)

The result is:

[1]

[1, 3]

[1, 3, 5]

As you can see from the results, when When the append_item function is called the next two times, the function parameter b is not initialized to [], but retains the value of the previous function call.

The reason why we get this result is that in Python, the default value of a function parameter is only initialized once when the function is defined.

Let’s look at an example to prove this feature of Python:

class Test(object):  
    def __init__(self):  
        print("Init Test")  
           
def arg_init(a, b = Test()):  
    print(a)  
arg_init(1)  
arg_init(3)  
arg_init(5)

The result is:

Init Test

1

3

5

From this example You can see the results of Note that the Test class is only instantiated once, which means that the default parameters have nothing to do with the number of function calls and are only initialized once when the function is defined.

Correct use of variable default parameters

For variable default parameters, we can use the following pattern to avoid the above unexpected results:

def append_item(a = 1, b = None):
    if b is None:
        b = []
    b.append(a)
    print b
     
append_item(a=1)
append_item(a=3)
append_item(a=5)

The result is:

[1]

[3 ]

[5]

Scope in Python

Python’s scope resolution order is Local, Enclosing, Global, Built-in, which means that the Python interpreter will parse variables according to this order.

Look at a simple example:

global_var = 0
def outer_func():
    outer_var = 1
     
    def inner_func():
        inner_var = 2
         
        print "global_var is :", global_var
        print "outer_var is :", outer_var
        print "inner_var is :", inner_var
         
    inner_func()
     
outer_func()

The result is:

global_var is : 0

outer_var is : 1

inner_var is : 2

In Python, about One thing to note about scope is that when assigning a value to a variable in a scope, Python will consider the variable to be a local variable of the current scope.

This is also relatively easy to understand. For the following code, var_func assigns a value to the num variable, so num here is a local variable in the var_func scope.

num = 0
def var_func():
    num = 1
    print "num is :", num
     
var_func()

Question 1

However, when we use variables in the following way, problems will arise:

num = 0
def var_func():
    print "num is :", num
    num = 1
     
var_func()

The results are as follows :

UnboundLocalError: local variable 'num' referenced before assignment

The reason why this error occurs is because we assigned a value to the num variable in var_func, so the Python interpreter will think that num is a local variable in the var_func scope, but when the code is executed to print " num is: ", num is still undefined when executing the num statement.

Question 2

The above error is relatively obvious, and there is also a more subtle error form as follows:

   
li = [1, 2, 3]
def foo():
    li.append(4)
    print li
foo()
def bar():
    li +=[5]
    print li
bar()

The result of the code is:

[ 1, 2, 3, 4]

UnboundLocalError: local variable 'li' referenced before assignment

In the foo function, according to Python’s scope resolution order, the global li variable is used in this function; but in the bar function, the li variable is Assignment, so li will be treated as a variable in the scope of bar.

For this problem of the bar function, you can use the global keyword.

li = [1, 2, 3]
def foo():
    li.append(4)
    print li
     
foo()
def bar():
    global li
    li +=[5]
    print li
     
bar()

Class attribute hiding

In Python, there are class attributes and instance attributes. Class attributes belong to the class itself and are shared by all class instances.

Class attributes can be accessed and modified through the class name or through the class instance. However, when an instance defines an attribute with the same name as the class, the class attribute is hidden.

Look at the following example:

class Student(object):
    books = ["Python", "JavaScript", "CSS"]
    def __init__(self, name, age):
        self.name = name
        self.age = age
    pass
     
wilber = Student("Wilber", 27)
print "%s is %d years old" %(wilber.name, wilber.age)
print Student.books
print wilber.books
wilber.books = ["HTML", "AngularJS"]
print Student.books
print wilber.books
del wilber.books
print Student.books
print wilber.books

The result of the code is as follows. At first, the wilber instance can directly access the books attribute of the class, but when the instance wilber defines an instance attribute named books, the books attribute of the wilber instance " The books attribute of the class is hidden; after the books attribute of the wilber instance is deleted, wilber.books corresponds to the books attribute of the class again.

Wilber is 27 years old
['Python', 'JavaScript', 'CSS']
['Python', 'JavaScript', 'CSS']
['Python', 'JavaScript', 'CSS']
['HTML', 'AngularJS']
['Python', 'JavaScript', 'CSS']
['Python', 'JavaScript', 'CSS']

When using inheritance in Python values, you should also pay attention to the hiding of class attributes. For a class, you can view all class attributes through the __dict__ attribute of the class.

When accessing a class attribute through the class name, it will first search for the __dict__ attribute of the class. If the class attribute is not found, it will continue to search for the parent class. However, if the subclass defines a class attribute with the same name as the parent class, the class attributes of the subclass will hide the class attributes of the parent class.

Look at an example:

class A(object):
    count = 1
     
class B(A):
    pass    
     
class C(A):
    pass        
     
print A.count, B.count, C.count      
B.count = 2
print A.count, B.count, C.count      
A.count = 3
print A.count, B.count, C.count     
print B.__dict__
print C.__dict__

The result is as follows. When class B defines the class attribute count, the count attribute of the parent class will be hidden

1 1 1
1 2 1
3 2 3
{'count': 2, '__module__': '__main__', '__doc__': None}
{'__module__': '__main__', '__doc__': None}

tuple is "variable"

在Python中，tuple是不可变对象，但是这里的不可变指的是tuple这个容器总的元素不可变（确切的说是元素的id），但是元素的值是可以改变的。

tpl = (1, 2, 3, [4, 5, 6])
print id(tpl)
print id(tpl[3])
tpl[3].extend([7, 8])
print tpl
print id(tpl)
print id(tpl[3])

代码结果如下，对于tpl对象，它的每个元素都是不可变的，但是tpl[3]是一个list对象。也就是说，对于这个tpl对象，id(tpl[3])是不可变的，但是tpl[3]确是可变的。

36764576
38639896
(1, 2, 3, [4, 5, 6, 7, 8])
36764576
38639896

Python的深浅拷贝

在对Python对象进行赋值的操作中，一定要注意对象的深浅拷贝，一不小心就可能踩坑了。

当使用下面的操作的时候，会产生浅拷贝的效果：

使用切片[:]操作

使用工厂函数（如list/dir/set）

使用copy模块中的copy()函数

使用copy模块里面的浅拷贝函数copy()：

import copy
will = ["Will", 28, ["Python", "C#", "JavaScript"]]
wilber = copy.copy(will)
print id(will)
print will
print [id(ele) for ele in will]
print id(wilber)
print wilber
print [id(ele) for ele in wilber]
will[0] = "Wilber"
will[2].append("CSS")
print id(will)
print will
print [id(ele) for ele in will]
print id(wilber)
print wilber
print [id(ele) for ele in wilber]

使用copy模块里面的深拷贝函数deepcopy()：

import copy
will = ["Will", 28, ["Python", "C#", "JavaScript"]]
wilber = copy.deepcopy(will)
print id(will)
print will
print [id(ele) for ele in will]
print id(wilber)
print wilber
print [id(ele) for ele in wilber]
will[0] = "Wilber"
will[2].append("CSS")
print id(will)
print will
print [id(ele) for ele in will]
print id(wilber)
print wilber
print [id(ele) for ele in wilber]

模块循环依赖

在Python中使用import导入模块的时候，有的时候会产生模块循环依赖，例如下面的例子，module_x模块和module_y模块相互依赖，运行module_y.py的时候就会产生错误。  

# module_x.py
import module_y
     
def inc_count():
    module_y.count += 1
    print module_y.count
     
     
# module_y.py
import module_x
count = 10
def run():
    module_x.inc_count()
     
run()

其实，在编码的过程中就应当避免循环依赖的情况，或者代码重构的过程中消除循环依赖。

当然，上面的问题也是可以解决的，常用的解决办法就是把引用关系搞清楚，让某个模块在真正需要的时候再导入（一般放到函数里面）。

对于上面的例子，就可以把module_x.py修改为如下形式，在函数内部导入module_y：

# module_x.py
def inc_count():
    import module_y
    module_y.count += 1