了解 Python 中的线程和多重处理：综合指南

介绍

在 Python 中，在优化应用程序性能时，尤其是当它们涉及并发或并行执行时，经常会讨论 线程 和 多处理 的概念。尽管术语有重叠，但这两种方法本质上是不同的。

本博客将有助于澄清线程和多处理的混淆，解释何时使用每个概念，并为每个概念提供相关示例。

---

线程与多处理：主要区别

在深入示例和用例之前，让我们概述一下主要区别：

• 线程：是指在单个进程中运行多个线程（进程的较小单元）。线程共享相同的内存空间，这使得它们变得轻量级。然而，Python 的全局解释器锁 (GIL) 限制了 CPU 密集型任务的线程的真正并行性。

• 多处理：涉及运行多个进程，每个进程都有自己的内存空间。进程比线程重，但可以实现真正的并行性，因为它们不共享内存。这种方法非常适合需要充分利用核心的 CPU 密集型任务。

---

什么是线程？

线程是一种在同一进程中同时运行多个任务的方法。这些任务由线程处理，它们是共享相同内存空间的独立的轻量级执行单元。线程有利于 I/O 密集型操作，例如文件读取、网络请求或数据库查询，这些操作中主程序花费大量时间等待外部资源。

何时使用线程

• 当您的程序受 I/O 限制时（例如，读/写文件、发出网络请求）。
• 当任务花费大量时间等待输入或输出操作时。
• 当您需要在单个进程中实现轻量级并发时。

示例：基本线程

import threading
import time

def print_numbers():
    for i in range(5):
        print(i)
        time.sleep(1)

def print_letters():
    for letter in ['a', 'b', 'c', 'd', 'e']:
        print(letter)
        time.sleep(1)

# Create two threads
t1 = threading.Thread(target=print_numbers)
t2 = threading.Thread(target=print_letters)

# Start both threads
t1.start()
t2.start()

# Wait for both threads to complete
t1.join()
t2.join()

print("Both threads finished execution.")

在上面的示例中，两个线程并发运行：一个打印数字，另一个打印字母。 sleep() 调用模拟 I/O 操作，程序可以在这些等待期间在线程之间切换。

线程问题：全局解释器锁 (GIL)

Python 的 GIL 是一种防止多个本机线程同时执行 Python 字节码的机制。它确保一次只有一个线程运行，即使进程中有多个线程处于活动状态。

此限制使得线程不适合需要真正并行性的 CPU 密集型任务，因为由于 GIL，线程无法充分利用多个内核。

---

什么是多重处理？

多处理允许您同时运行多个进程，其中每个进程都有自己的内存空间。由于进程不共享内存，因此没有 GIL 限制，允许在多个 CPU 内核上真正并行执行。多重处理非常适合需要最大化 CPU 使用率的 CPU 密集型任务。

何时使用多重处理

• 当您的程序受 CPU 限制时（例如，执行繁重的计算、数据处理）。
• 当您需要真正的并行性而不需要内存共享时。
• 当您想要同时运行独立任务的多个实例时。

示例：基本多重处理

import multiprocessing
import time

def print_numbers():
    for i in range(5):
        print(i)
        time.sleep(1)

def print_letters():
    for letter in ['a', 'b', 'c', 'd', 'e']:
        print(letter)
        time.sleep(1)

if __name__ == "__main__":
    # Create two processes
    p1 = multiprocessing.Process(target=print_numbers)
    p2 = multiprocessing.Process(target=print_letters)

    # Start both processes
    p1.start()
    p2.start()

    # Wait for both processes to complete
    p1.join()
    p2.join()

    print("Both processes finished execution.")

在此示例中，两个单独的进程同时运行。与线程不同，每个进程都有自己的内存空间，并且独立执行，不受GIL的干扰。

多处理中的内存隔离

线程和多处理之间的一个关键区别是进程不共享内存。虽然这确保了进程之间没有干扰，但这也意味着它们之间共享数据需要特殊的机制，例如多处理模块提供的 Queue、Pipe 或 Manager 对象。

---

线程与多处理：选择正确的工具

现在我们了解了这两种方法的工作原理，让我们根据任务类型详细说明何时选择线程或多处理：

Use Case	Type	Why?
Network requests, I/O-bound tasks (file read/write, DB calls)	Threading	Multiple threads can handle I/O waits concurrently.
CPU-bound tasks (data processing, calculations)	Multiprocessing	True parallelism is possible by utilizing multiple cores.
Task requires shared memory or lightweight concurrency	Threading	Threads share memory and are cheaper in terms of resources.
Independent tasks needing complete isolation (e.g., separate processes)	Multiprocessing	Processes have isolated memory, making them safer for independent tasks.

---

Performance Considerations

Threading Performance

Threading excels in scenarios where the program waits on external resources (disk I/O, network). Since threads can work concurrently during these wait times, threading can help boost performance.

However, due to the GIL, CPU-bound tasks do not benefit much from threading because only one thread can execute at a time.

Multiprocessing Performance

Multiprocessing allows true parallelism by running multiple processes across different CPU cores. Each process runs in its own memory space, bypassing the GIL and making it ideal for CPU-bound tasks.

However, creating processes is more resource-intensive than creating threads, and inter-process communication can slow things down if there's a lot of data sharing between processes.

---

A Practical Example: Threading vs. Multiprocessing for CPU-bound Tasks

Let's compare threading and multiprocessing for a CPU-bound task like calculating the sum of squares for a large list.

Threading Example for CPU-bound Task

import threading

def calculate_squares(numbers):
    result = sum([n * n for n in numbers])
    print(result)

numbers = range(1, 10000000)
t1 = threading.Thread(target=calculate_squares, args=(numbers,))
t2 = threading.Thread(target=calculate_squares, args=(numbers,))

t1.start()
t2.start()

t1.join()
t2.join()

Due to the GIL, this example will not see significant performance improvements over a single-threaded version because the threads can't run simultaneously for CPU-bound operations.

Multiprocessing Example for CPU-bound Task

import multiprocessing

def calculate_squares(numbers):
    result = sum([n * n for n in numbers])
    print(result)

if __name__ == "__main__":
    numbers = range(1, 10000000)
    p1 = multiprocessing.Process(target=calculate_squares, args=(numbers,))
    p2 = multiprocessing.Process(target=calculate_squares, args=(numbers,))

    p1.start()
    p2.start()

    p1.join()
    p2.join()

In the multiprocessing example, you'll notice a performance boost since both processes run in parallel across different CPU cores, fully utilizing the machine's computational resources.

---

Conclusion

Understanding the difference between threading and multiprocessing is crucial for writing efficient Python programs. Here’s a quick recap:

• Use threading for I/O-bound tasks where your program spends a lot of time waiting for resources.
• Use multiprocessing for CPU-bound tasks to maximize performance through parallel execution.

Knowing when to use which approach can lead to significant performance improvements and efficient use of resources.

以上是了解 Python 中的线程和多重处理：综合指南的详细内容。更多信息请关注PHP中文网其他相关文章！