Understanding threads in Python

We will see some examples of using threads in Python and how to avoid thread competition.

You should run the following example multiple times so that you can notice that threads are unpredictable and threads produce different results each time they are run. Disclaimer: Forget what you've heard about the GIL from here on out, because the GIL doesn't affect what I'm trying to show.

Example 1

We are going to request five different URLs:

Single thread

import time
import urllib2

def get_responses():
    urls = [
        'http://www.google.com',
        'http://www.amazon.com',
        'http://www.ebay.com',
        'http://www.alibaba.com',
        'http://www.reddit.com'
    ]
    start = time.time()
    for url in urls:
        print url
        resp = urllib2.urlopen(url)
        print resp.getcode()
    print "Elapsed time: %s" % (time.time()-start)

get_responses()

The output is:

http://www.google.com 200
http://www.amazon.com 200
http://www.ebay.com 200
http://www.alibaba.com 200
http://www.reddit.com 200
Elapsed time: 3.0814409256

explain:

• The urls are requested in order

• 
  

• # Unless the CPU gets a response from one url, it will not request the next url

• 
  

• Network requests take a long time, so the CPU remains idle while waiting for the return of the network request.

Multithreading

import urllib2
import time
from threading import Thread

class GetUrlThread(Thread):
    def __init__(self, url):
        self.url = url 
        super(GetUrlThread, self).__init__()

    def run(self):
        resp = urllib2.urlopen(self.url)
        print self.url, resp.getcode()

def get_responses():
    urls = [
        'http://www.google.com', 
        'http://www.amazon.com', 
        'http://www.ebay.com', 
        'http://www.alibaba.com', 
        'http://www.reddit.com'
    ]
    start = time.time()
    threads = []
    for url in urls:
        t = GetUrlThread(url)
        threads.append(t)
        t.start()
    for t in threads:
        t.join()
    print "Elapsed time: %s" % (time.time()-start)

get_responses()

Output:

http://www.reddit.com 200
http://www.google.com 200
http://www.amazon.com 200
http://www.alibaba.com 200
http://www.ebay.com 200
Elapsed time: 0.689890861511

explain:

• Realized the improvement in program execution time

• 
  

• We wrote a multi-threaded program to reduce the waiting time of the CPU. When we are waiting for a network request in one thread to return, the CPU can switch to other threads to make network requests in other threads.

• 
  

• We expect a thread to process a url, so we pass a url when instantiating the thread class.

• 
  

• Thread running means executing the run() method in the class.

• 
  

• However we think each thread must execute run().

• 
  

• Create a thread for each URL and call the start() method, which tells the CPU to execute the run() method in the thread.

• 
  

• We hope to calculate the time spent when all threads have completed execution, so we call the join() method.

• 
  

• join()You can notify the main thread to wait for this thread to end before executing the next instruction.

• 
  

• We call the join() method for each thread, so we calculate the running time after all threads have completed execution.

About the thread:

• The cpu may not execute the run() method immediately after calling start().

• 
  

• You cannot determine the execution order of run() among different threads.

• 
  

• For a single thread, it is guaranteed that the statements in the run() method are executed in order.

• 
  

• This is because the url within the thread will be requested first, and then the returned result will be printed.

Example 2

We will use a program to demonstrate resource competition between multiple threads and fix this problem.

from threading import Thread

#define a global variable
some_var = 0 

class IncrementThread(Thread):
    def run(self):
        #we want to read a global variable
        #and then increment it
        global some_var
        read_value = some_var
        print "some_var in %s is %d" % (self.name, read_value)
        some_var = read_value + 1 
        print "some_var in %s after increment is %d" % (self.name, some_var)

def use_increment_thread():
    threads = []
    for i in range(50):
        t = IncrementThread()
        threads.append(t)
        t.start()
    for t in threads:
        t.join()
    print "After 50 modifications, some_var should have become 50"
    print "After 50 modifications, some_var is %d" % (some_var,)

use_increment_thread()

Run this program multiple times and you will see a variety of different results.

explain:

• There is a global variable that all threads want to modify.

• 
  

• All threads should add 1 to this global variable.

• 
  

• With 50 threads, the final value should become 50, but it doesn't.

Why didn’t it reach 50?

• When some_var is 15, thread t1 reads some_var, and at this moment the cpu gives control to another threadt2.

• 
  

• t2The some_var read by the thread is also 15

• 
  

• ##

  t1 and t2 both add some_var to 16

• 
  

• At that time, what we expected was that

  t1 t2 two threads would make some_var + 2 become 17

• 
  

• There is competition for resources here.

• 
  

• The same situation may also occur between other threads, so the final result may be less than

  50.

Solving resource competition

from threading import Lock, Thread
lock = Lock()
some_var = 0 

class IncrementThread(Thread):
    def run(self):
        #we want to read a global variable
        #and then increment it
        global some_var
        lock.acquire()
        read_value = some_var
        print "some_var in %s is %d" % (self.name, read_value)
        some_var = read_value + 1 
        print "some_var in %s after increment is %d" % (self.name, some_var)
        lock.release()

def use_increment_thread():
    threads = []
    for i in range(50):
        t = IncrementThread()
        threads.append(t)
        t.start()
    for t in threads:
        t.join()
    print "After 50 modifications, some_var should have become 50"
    print "After 50 modifications, some_var is %d" % (some_var,)

use_increment_thread()

Running this program again achieved the results we expected.

explain:

• Lock is used to prevent race conditions

• 
  

• # If thread

  t1 acquires the lock before performing some operations. Other threads will not perform the same operation beforet1releases the Lock

• 
  

• 我们想要确定的是一旦线程t1已经读取了some_var，直到t1完成了修改some_var，其他的线程才可以读取some_var

• 
  

• 这样读取和修改some_var成了逻辑上的原子操作。

　　实例3

　　让我们用一个例子来证明一个线程不能影响其他线程内的变量（非全局变量）。

　　time.sleep()可以使一个线程挂起，强制线程切换发生。

from threading import Thread
import time

class CreateListThread(Thread):
    def run(self):
        self.entries = []
        for i in range(10):
            time.sleep(1)
            self.entries.append(i)
        print self.entries

def use_create_list_thread():
    for i in range(3):
        t = CreateListThread()
        t.start()

use_create_list_thread()

　　运行几次后发现并没有打印出争取的结果。当一个线程正在打印的时候，cpu切换到了另一个线程，所以产生了不正确的结果。我们需要确保print self.entries是个逻辑上的原子操作，以防打印时被其他线程打断。

　　我们使用了Lock()，来看下边的例子。

from threading import Thread, Lock
import time

lock = Lock()

class CreateListThread(Thread):
    def run(self):
        self.entries = []
        for i in range(10):
            time.sleep(1)
            self.entries.append(i)
        lock.acquire()
        print self.entries
        lock.release()

def use_create_list_thread():
    for i in range(3):
        t = CreateListThread()
        t.start()

use_create_list_thread()

　　这次我们看到了正确的结果。证明了一个线程不可以修改其他线程内部的变量（非全局变量）。

　　原文出处： Akshar Raaj

The above is the detailed content of Understanding threads in Python. For more information, please follow other related articles on the PHP Chinese website!