How to Achieve Non-Blocking Subprocess Pipe Reads in Python?

How to Perform Non-Blocking Reads on Subprocess Pipes in Python

When working with the subprocess module in Python, it can be necessary to perform non-blocking reads on the output stream of a subprocess. This ensures that the program doesn't block when reading from a process that may not have any data available.

Traditional Blocking Reads

Typically, the following code would be used to read from the standard output of a subprocess:

p = subprocess.Popen('myprogram.exe', stdout = subprocess.PIPE)
output_str = p.stdout.readline()

However, this approach blocks the execution of the program until data is available on the standard output.

Non-Blocking Reads

To implement non-blocking reads, one common method is to use the Queue class from the Python queue module. Here's an example:

import sys
from subprocess import PIPE, Popen
from threading  import Thread
try:
    from queue import Queue, Empty
except ImportError:
    from Queue import Queue, Empty  # python 2.x

ON_POSIX = 'posix' in sys.builtin_module_names

def enqueue_output(out, queue):
    for line in iter(out.readline, b''):
        queue.put(line)
    out.close()

p = Popen(['myprogram.exe'], stdout=PIPE, bufsize=1, close_fds=ON_POSIX)
q = Queue()
t = Thread(target=enqueue_output, args=(p.stdout, q))
t.daemon = True # thread dies with the program
t.start()

# ... do other things here

# read line without blocking
try:  line = q.get_nowait() # or q.get(timeout=.1)
except Empty:
    print('no output yet')
else: # got line
    # ... do something with line

In this code, the enqueue_output function runs in a separate thread and不断地将 output从subprocess stdout放入队列中。主线程可以随时调用 q.get_nowait() 来检查队列中是否有数据.如果没有数据，它将引发 Empty 异常，而如果成功，它将返回提取的行。

The above is the detailed content of How to Achieve Non-Blocking Subprocess Pipe Reads in Python?. For more information, please follow other related articles on the PHP Chinese website!