An introduction to multi-threading in Python

Multi-threading can be simply understood as executing multiple tasks at the same time. This article will share with you a detailed example of Python multi-threading Threading beginner tutorial. Friends who are interested can learn together

1.1 What is multi-threading

Multithreading can simply be understood as executing multiple tasks at the same time.

Both multi-process and multi-threading can perform multiple tasks, and threads are part of the process. The characteristic of threads is that they can share memory and variables between threads, and consume less resources (however, in the Unix environment, the difference in resource scheduling consumption between multi-process and multi-thread is not obvious, and Unix scheduling is faster). The disadvantage is the synchronization and acceleration between threads. Locks are more troublesome.

1.2 Add thread Thread

Import module

##

import threading

Get activated Number of threads

threading.active_count()

View all thread information

threading.enumerate()

View currently running threads

threading.current_thread()

Add a thread,

threading.Thread()Receive parameter target represents the task to be completed by this thread, you need to define it yourself

def thread_job():
  print('This is a thread of %s' % threading.current_thread())
def main():
  thread = threading.Thread(target=thread_job,)  # 定义线程 
  thread.start() # 让线程开始工作
  if __name__ == '__main__':
  main()

1.3 join function

Because the threads are running at the same time, using the join function allows the thread to complete before proceeding to the next step, that is, blocking the calling thread. , until all tasks in the queue are processed.

import threading
import time
def thread_job():
  print('T1 start\n')
  for i in range(10):
    time.sleep(0.1)
  print('T1 finish\n')
def T2_job():
  print('T2 start\n')
  print('T2 finish\n')
def main():
  added_thread=threading.Thread(target=thread_job,name='T1')
  thread2=threading.Thread(target=T2_job,name='T2')
  added_thread.start()
  #added_thread.join()
  thread2.start()
  #thread2.join()
  print('all done\n')
if __name__=='__main__':
   main()

The example is shown above. When the join function is not used, the result is as shown below:

When the join function is executed, T2 will be run only after T1 has finished running, and then run print ('all done')

1.4 Queue for storing process results

Queue is a thread-safe queue (FIFO) implementation in the Python standard library, which provides a first-in-first-out data structure suitable for multi-threaded programming. , that is, a queue, used to transfer information between producer and consumer threads

(1) Basic FIFO queue

 class queue.Queue(maxsize=0)

maxsize is an integer, Indicates the upper limit of the number of data that can be stored in the queue. When the upper limit is reached, insertion will cause blocking until the data in the queue is consumed. If maxsize is less than or equal to 0, there is no limit on the queue size

(2) LIFO queue last in first out

class queue.LifoQueue(maxsize=0)

(3) Priority queue

class queue.PriorityQueue(maxsize=0)

in the video I don’t understand the code very well

import threading
import time
from queue import Queue
def job(l,q):
  for i in range(len(l)):
    l[i]=l[i]**2
  q.put(l)
def multithreading():
  q=Queue()
  threads=[]
  data=[[1,2,3],[3,4,5],[4,5,6],[5,6,7]]
  for i in range(4):
    t=threading.Thread(target=job,args=(data[i],q))
    t.start()
    threads.append(t)
  for thread in threads:
    thread.join()
  results=[]
  for _ in range(4):
    results.append(q.get())
  print(results)
if __name__=='__main__':
   multithreading()

The running result is as follows

## 1.5 GIL is not necessarily efficient

Global Interpreter Lock Global interpreter lock, the execution of python is controlled by the python virtual machine (also called the interpreter main loop), and the control of GIL controls the python virtual machine Access ensures that only one thread is running in the interpreter at any time. In a multi-threaded environment, the python virtual machine executes in the following manner:

1. Set up GIL

2. Switch to a thread to run

3. Run:

a. Specify the number of bytecode instructions, or

b. The thread actively gives up control (you can call time.sleep(0))

4. Set the thread For sleep state

5. Unlock GIL

6. Repeat 1-5

When calling external code (such as C/C++ extension functions), the GIL will be Locked until the end of this function (since no python bytecode is run during this period, no thread switching will be performed).

The following is the code example in the video. It expands a number by 4 times, divides it into the normal method and allocates it to 4 threads. It is found that the time consumption is not much different.

import threading
from queue import Queue
import copy
import time
def job(l, q):
  res = sum(l)
  q.put(res)
def multithreading(l):
  q = Queue()
  threads = []
  for i in range(4):
    t = threading.Thread(target=job, args=(copy.copy(l), q), name='T%i' % i)
    t.start()
    threads.append(t)
  [t.join() for t in threads]
  total = 0
  for _ in range(4):
    total += q.get()
  print(total)
def normal(l):
  total = sum(l)
  print(total)
if __name__ == '__main__':
  l = list(range(1000000))
  s_t = time.time()
  normal(l*4)
  print('normal: ',time.time()-s_t)
  s_t = time.time()
  multithreading(l)
  print('multithreading: ', time.time()-s_t)

The running result is:

1.6 线程锁 Lock

如果线程1得到了结果，想要让线程2继续使用1的结果进行处理，则需要对1lock，等到1执行完，再开始执行线程2。一般来说对share memory即对共享内存进行加工处理时会用到lock。

import threading
def job1():
  global A, lock #全局变量
  lock.acquire() #开始lock
  for i in range(10):
    A += 1
    print('job1', A)
  lock.release() #释放
def job2(): 
  global A, lock
  lock.acquire()
  for i in range(10):
    A += 10
    print('job2', A)
  lock.release()
if __name__ == '__main__':
  lock = threading.Lock()
  A = 0
  t1 = threading.Thread(target=job1)
  t2 = threading.Thread(target=job2)
  t1.start()
  t2.start()
  t1.join()
  t2.join()

运行结果如下所示：

总结

The above is the detailed content of An introduction to multi-threading in Python. For more information, please follow other related articles on the PHP Chinese website!