What is the method of debugging in Python?

Logging is a must

If you write an application without some kind of logging setup, you will eventually regret it. Without any logs in the application, it is difficult to troubleshoot any errors. Fortunately, in Python, setting up a basic logger is very simple:

import logging
logging.basicConfig(
    filename='application.log',
    level=logging.WARNING,
    format= '[%(asctime)s] {%(pathname)s:%(lineno)d} %(levelname)s - %(message)s',
    datefmt='%H:%M:%S'
)

logging.error("Some serious error occurred.")
logging.warning('Function you are using is deprecated.')

That's all you need to start writing logs to a file, which will look like this (you can use logging.getLoggerClass().root.handlers[0].baseFilenameFind file path):

[12:52:35] {<stdin>:1} ERROR - Some serious error occurred.
[12:52:35] {<stdin>:1} WARNING - Function you are using is deprecated.

This setup seems to be good enough (as is usually the case), but the configuration is well-formatted Well-organized, readable logs can make your life easier. One way to improve and extend your configuration is to use an .ini or .yaml file that the logger reads. For example, you could do the following in the configuration:

version: 1
disable_existing_loggers: true

formatters:
  standard:
    format: "[%(asctime)s] {%(pathname)s:%(lineno)d} %(levelname)s - %(message)s"
    datefmt: '%H:%M:%S'

handlers:
  console:  # handler which will log into stdout
    class: logging.StreamHandler
    level: DEBUG
    formatter: standard  # Use formatter defined above
    stream: ext://sys.stdout
  file:  # handler which will log into file
    class: logging.handlers.RotatingFileHandler
    level: WARNING
    formatter: standard  # Use formatter defined above
    filename: /tmp/warnings.log
    maxBytes: 10485760 # 10MB
    backupCount: 10
    encoding: utf8

root:  # Loggers are organized in hierarchy - this is the root logger config
  level: ERROR
  handlers: [console, file]  # Attaches both handler defined above

loggers:  # Defines descendants of root logger
  mymodule:  # Logger for "mymodule"
    level: INFO
    handlers: [file]  # Will only use "file" handler defined above
    propagate: no  # Will not propagate logs to "root" logger

Having this kind of extensive configuration in Python code would be difficult to navigate, edit, and maintain. Saving the content in a YAML file makes it easier to set up and tune multiple loggers with very specific settings like the one above.

If you're wondering where all these configuration fields come from, these are documented in the official documentation, and most of them are just keyword parameters, as shown in the first example.

So now we have the configuration in the file, meaning we need to load it somehow. The easiest way is to use a YAML file:

import yaml
from logging import config

with open("config.yaml", 'rt') as f:
    config_data = yaml.safe_load(f.read())
    config.dictConfig(config_data)

Python logger doesn't actually support YAML files directly, but it does support dictionary configuration, which can be done using yaml.safe_load Easily create dictionary configurations from YAML. If you prefer to use the old .ini file, then I just want to point out that according to the official documentation, using dictionary configuration is the recommended approach for new applications. For more examples, check out the official logging manual.

Log Decorator

Continuing with the previous logging techniques, you may encounter situations where you need to record some error function calls. Instead of modifying the body of said function, you can use a logging decorator, which will log each function call with a specific log level and optional message. Let's look at decorators:

from functools import wraps, partial
import logging

def attach_wrapper(obj, func=None):  # Helper function that attaches function as attribute of an object
    if func is None:
        return partial(attach_wrapper, obj)
    setattr(obj, func.__name__, func)
    return func

def log(level, message):  # Actual decorator
    def decorate(func):
        logger = logging.getLogger(func.__module__)  # Setup logger
        formatter = logging.Formatter(
            '%(asctime)s - %(name)s - %(levelname)s - %(message)s')
        handler = logging.StreamHandler()
        handler.setFormatter(formatter)
        logger.addHandler(handler)
        log_message = f"{func.__name__} - {message}"

        @wraps(func)
        def wrapper(*args, **kwargs):  # Logs the message and before executing the decorated function
            logger.log(level, log_message)
            return func(*args, **kwargs)

        @attach_wrapper(wrapper)  # Attaches "set_level" to "wrapper" as attribute
        def set_level(new_level):  # Function that allows us to set log level
            nonlocal level
            level = new_level

        @attach_wrapper(wrapper)  # Attaches "set_message" to "wrapper" as attribute
        def set_message(new_message):  # Function that allows us to set message
            nonlocal log_message
            log_message = f"{func.__name__} - {new_message}"

        return wrapper
    return decorate

# Example Usage
@log(logging.WARN, "example-param")
def somefunc(args):
    return args

somefunc("some args")

somefunc.set_level(logging.CRITICAL)  # Change log level by accessing internal decorator function
somefunc.set_message("new-message")  # Change log message by accessing internal decorator function
somefunc("some args")

Needless to say, this may take a little while to wrap your head around (you may just want to copy-paste and use it). The idea here is that the log function accepts arguments and provides them to the inner wrapper function. Then, make these parameters adjustable by adding accessor functions attached to the decorator. As for the functools.wraps decorator - if we don't use it here, the name of the function ( func.__name__) will be overwritten by the name of the decorator. But this is a problem because we want to print the names. This can be solved by functools.wrapscopying the function name, docstring, and argument list to the decorator function.

Anyway, this is the output of the above code. Pretty neat, right?

2020-05-01 14:42:10,289 - __main__ - WARNING - somefunc - example-param
2020-05-01 14:42:10,289 - __main__ - CRITICAL - somefunc - new-message

__repr__More readable logs

A simple improvement to the code to make it easier to debug is to add the __repr__ method to the class. If you're not familiar with this method, all it does is return a string representation of the class instance. The best practice with the __repr__ method is to output text that can be used to recreate the instance. For example:

class Circle:
    def __init__(self, x, y, radius):
        self.x = x
        self.y = y
        self.radius = radius

    def __repr__(self):
        return f"Rectangle({self.x}, {self.y}, {self.radius})"

...
c = Circle(100, 80, 30)
repr(c)
# Circle(100, 80, 30)

If the object is undesirable or impossible as shown above, then a good alternative is to use <...>, such as <_io.TextIOWrapper name='somefile.txt' mode='w' encoding='UTF-8'>.

In addition to __repr__, it is also a good idea to implement the __str__ method, which is used by default when calling print(instance) method. Using these two methods, you can get a lot of information just by printing the variables.

__missing__Dunder method of dictionary

If you need to implement a custom dictionary class for any reason, then when you try to access some keys that don't actually exist, Some bugs may occur due to KeyError. To avoid having to look around in your code for a missing key, you can implement a special __missing__ method that is called every time a KeyError is raised.

class MyDict(dict):
    def __missing__(self, key):
        message = f'{key} not present in the dictionary!'
        logging.warning(message)
        return message  # Or raise some error instead

The above implementation is very simple, just returns and logs the message with the missing key, but you can also log other valuable information to provide you with information about errors in your code for more context.

调试崩溃的应用程序

如果你的应用程序在你有机会看到其中发生了什么之前崩溃，你可能会发现这个技巧非常有用。

-i使用参数-i ( python3 -i app.py)运行应用程序会导致它在程序退出后立即启动交互式 shell。此时你可以检查变量和函数。

如果这还不够好，可以使用更大的hammer-pdb-Python调试器。pdb有相当多的特性，可以保证文章的独立性。但这里是一个例子和最重要的部分概要。让我们先看看我们的小崩溃脚本：

# crashing_app.py
SOME_VAR = 42

class SomeError(Exception):
    pass

def func():
    raise SomeError("Something went wrong...")

func()

现在，如果我们使用-i参数运行它，我们就有机会调试它：

# Run crashing application
~ $ python3 -i crashing_app.py
Traceback (most recent call last):
  File "crashing_app.py", line 9, in <module>
    func()
  File "crashing_app.py", line 7, in func
    raise SomeError("Something went wrong...")
__main__.SomeError: Something went wrong...
>>> # We are interactive shell
>>> import pdb
>>> pdb.pm()  # start Post-Mortem debugger
> .../crashing_app.py(7)func()
-> raise SomeError("Something went wrong...")
(Pdb) # Now we are in debugger and can poke around and run some commands:
(Pdb) p SOME_VAR  # Print value of variable
42
(Pdb) l  # List surrounding code we are working with
  2
  3   class SomeError(Exception):
  4       pass
  5
  6   def func():
  7  ->     raise SomeError("Something went wrong...")
  8
  9   func()
[EOF]
(Pdb)  # Continue debugging... set breakpoints, step through the code, etc.

上面的调试会话非常简单地展示了如何使用pdb。程序终止后，我们进入交互式调试会话。首先，我们导入pdb并启动调试器。此时，我们可以使用所有pdb命令。作为上面的示例，我们使用p命令打印变量，使用l命令打印列表代码。大多数情况下，你可能希望设置断点，你可以使用b LINE_NO来设置断点，并运行程序，直到达到断点（c），然后继续使用s单步执行函数，也可以使用w打印堆栈轨迹。有关命令的完整列表，你可以转到官方pdb文档。

检查堆栈轨迹

例如，假设你的代码是在远程服务器上运行的Flask或Django应用程序，你无法在其中获得交互式调试会话。在这种情况下，你可以使用traceback和sys包来了解代码中的错误：

import traceback
import sys

def func():
    try:
        raise SomeError("Something went wrong...")
    except:
        traceback.print_exc(file=sys.stderr)

运行时，上面的代码将打印引发的最后一个异常。除了打印例外，你还可以使用traceback包打印堆栈轨迹（traceback.print_stack()）或提取原始堆栈帧，对其进行格式化并进一步检查（traceback.format_list(traceback.extract_stack())）。

在调试期间重新加载模块

有时，你可能正在调试或试验交互式shell中的某些函数，并对其进行频繁更改。为了使运行/测试和修改的循环更容易，你可以运行importlib.reload(module)以避免每次更改后重新启动交互会话：

>>> import func from module
>>> func()
"This is result..."

# Make some changes to "func"
>>> func()
"This is result..."  # Outdated result
>>> from importlib import reload; reload(module)  # Reload "module" after changes made to "func"
>>> func()
"New result..."

这个技巧更多的是关于效率而不是调试。能够跳过一些不必要的步骤，使你的工作流程更快、更高效，这总是很好的。通常，不时地重新加载模块是一个好主意，因为它可以帮助你避免调试同时已经修改过多次的代码。

The above is the detailed content of What is the method of debugging in Python?. For more information, please follow other related articles on the PHP Chinese website!