python thread pool threadpool usage articles

This article mainly introduces the use of python thread pool threadpool in detail. It has certain reference value. Interested friends can refer to

I am currently working on a video device management project. , devices include (camera, DVR, NVR, etc.), including device information completion, device status push, device stream address push, etc. If a large number of devices are imported at the same time, if a single thread is used for device detection, then due to the large number of devices If there are too many, it will bring a large delay, so consider multi-threading to handle this problem.

You can use python language to implement the thread pool yourself, or you can use the third-party package threadpool thread pool package. This topic mainly introduces the use of threadpool and its specific implementation.

1. Installation

Use installation:

pip installthreadpool

2 , use

(1) Introduce the threadpool module
(2) Define the thread function
(3) Create a thread pool threadpool.ThreadPool()
(4) Create a thread pool that is required The task to be processed is threadpool.makeRequests()
(5) Put the multiple created tasks into the thread pool, threadpool.putRequest
(6) Wait until all tasks are processed theadpool.pool()

import threadpool 
def ThreadFun(arg1,arg2): 
 pass 
def main(): 
 device_list=[object1,object2,object3......,objectn]#需要处理的设备个数 
 task_pool=threadpool.ThreadPool(8)#8是线程池中线程的个数 
 request_list=[]#存放任务列表 
 #首先构造任务列表 
 for device in device_list: 
 request_list.append(threadpool.makeRequests(ThreadFun,[((device, ), {})])) 
 #将每个任务放到线程池中，等待线程池中线程各自读取任务，然后进行处理，使用了map函数，不了解的可以去了解一下。 
 map(task_pool.putRequest,request_list) 
 #等待所有任务处理完成，则返回，如果没有处理完，则一直阻塞 
 task_pool.poll() 
if __name__=="__main__": 
 main()

The above is a specific thread pool usage process
The specific definition of threadpool is as follows:

class ThreadPool: 
 """A thread pool, distributing work requests and collecting results. 
 
 See the module docstring for more information. 
 
 """ 
 def __init__(self, num_workers, q_size=0, resq_size=0, poll_timeout=5): 
 pass 
 def createWorkers(self, num_workers, poll_timeout=5): 
 pass 
 def dismissWorkers(self, num_workers, do_join=False): 
 pass 
 def joinAllDismissedWorkers(self): 
 pass 
 def putRequest(self, request, block=True, timeout=None): 
 pass 
 def poll(self, block=False): 
 pass 
 def wait(self): 
 pass

The next section will introduce the above entire process and each function in detail: python thread pool threadpool (implementation)

Related recommendations:

Implementation of python thread pool threadpool

The above is the detailed content of python thread pool threadpool usage articles. For more information, please follow other related articles on the PHP Chinese website!