The difference between python thread pool and multi-threading

The difference between python thread pool and multi-threading: 1. Threads travel under the process; 2. Threads travel under the process; 3. A process can contain multiple threads; 4. Data is difficult to share between different processes ;5. Processes consume more computer resources than threads.

1. Threads and multi-threads

Process: When a program is executed, it can be called a process, which includes the running program and the memory and system resources used by the program. A process is composed of multiple threads.

Thread: A thread is an execution stream in the program. Each thread has its own private register, and the code area It is shared. Different threads can execute the same function.

Multiple threads: Multi-threading means that a program contains multiple execution streams, that is, a program can run multiple different threads at the same time to perform different tasks, allowing a single program to create multiple parallel execution threads to complete their respective tasks.

The biggest benefit of multi-threading: Improve CPU utilization (especially suitable for I/O-intensive programs, the speed improvement is particularly obvious)

The difference between processes and threads:

Be a Simple metaphor: process = train, thread = carriage

Threads travel under the process (a simple carriage cannot run)

A process can contain multiple threads (a train can have multiple Carriage)

It is difficult to share data between different processes (it is difficult for passengers on one train to change to another train, such as station transfer)

It is very difficult to share data between different threads in the same process Easy to share (it is easy to change from carriage A to carriage B)

Processes consume more computer resources than threads (using multiple trains consumes more resources than multiple carriages)

Inter-process They will not affect each other. If one thread hangs up, the entire process will hang up (one train will not affect another train, but if the middle carriage of a train catches fire, it will affect all carriages)

The process can be expanded to multiple machines, and the process is suitable for up to multiple cores (different trains can run on multiple tracks, and the carriages of the same train cannot run on different tracks)

The memory address used by the process can be locked , that is, when a thread uses some shared memory, other threads must wait for it to end before they can use this piece of memory. (For example, the bathroom on the train) - "Mutex Lock"

The memory address used by the process can limit the usage (for example, a restaurant on the train, only a maximum number of people are allowed to enter, if it is full, you need to wait at the door, Wait for someone to come out before entering)-"Semaphore"

2. Multi-threading threading class

The threading class is the most commonly used multi-threading module. The method of creating a thread is as follows:

threading.Thread(group=None, target=None, name=None, args=(), kwargs={}, *, 
daemon=None)

group: The default is None.

target: The name of the function to be executed, remember not to include the parentheses of the function.

name: The name of the thread, the default is 'Thread-N' Form.

args: The parameter tuple of the callable object passed in the parameter target.

kwargs: The keyword argument dictionary of the callable object passed in the parameter target.

daemon: daemon mode attribute, the default is None.

Important methods of thread objects:

start(): Start the thread. It will make the run() method in an independent is called in the thread.

run(): This method represents thread activity.

join(timeout=None): Let the current caller thread wait until the thread ends.

daemon: Indicates whether the thread is a daemon thread, True or False.

Create a multi-threaded instance:

import random
import threading
import time
def awesome_function(name):
wait_time = random.randint(1, 10)
print('current thread name is :{0} and wait {1} s'.format(name, 
wait_time))
time.sleep(wait_time)
print('thread {} finished.'.format(name))
if __name__ == '__main__':
for i in range(0, 3):
t = threading.Thread(target=awesome_function, args=(i,))
t.start()

You can first look at the running results I recorded:

The above example starts 3 threads, and 3 threads execute tasks concurrently. The thread that completes the task first (the one with the shortest time.sleep) outputs the result first.

3. A better-used thread pool class ThreadPoolExecutor

Starting a new thread is very expensive because it involves interaction with the operating system. In this case, using a thread pool can greatly improve program performance, especially when a large number of programs need to be created with a short lifetime. When using threads, you should consider using a thread pool.

The thread pool creates a large number of idle threads when the system starts. As long as the program submits a function to the thread pool, the thread pool will start an idle thread for execution. It. When the execution of the function ends, the thread will not die, but will return to the thread pool again and become idle, waiting for the next function. At the same time, using the thread pool can effectively control the number of concurrent threads in the system. When the system contains a large number of concurrent threads, the system performance will drop sharply and even cause the Python interpreter to crash. The maximum number of threads parameter of the thread pool can control the number of concurrent threads in the system not to exceed this number.

The thread pool currently in mainstream use is ThreadPoolExecutor in the concurrent.futures module:

import random
import time
from concurrent.futures import ThreadPoolExecutor
def awesome_function(name):
wait_time = random.randint(1, 10)
print('current thread name is :{0} and wait {1} s'.format(name, 
wait_time))
time.sleep(wait_time)
print('thread {} finished.'.format(name))
if __name__ == '__main__':
with ThreadPoolExecutor(max_workers=3) as t:
for i in range(0, 3):
t.submit(awesome_function, i)

The running results are recorded as follows:

Create a thread pool object with a maximum capacity of 3 and submit it through submit The executed function is put into the thread pool. If a thread in the thread pool (thread 2) When the execution is completed, the idle thread (thread 3) is put into the pool, and so on, until all threads are completed, the program ends.

The above is the detailed content of The difference between python thread pool and multi-threading. For more information, please follow other related articles on the PHP Chinese website!