How Can I Achieve Non-Blocking Reads from a Subprocess's Standard Output?

Non-Blocking Reads for Subprocess Standard Output

When utilizing the subprocess module to initiate subprocesses and connect to their standard output streams, it is essential to perform non-blocking reads to maintain program responsiveness. This article delves into techniques to achieve non-blocking reads on subprocess standard outputs or assess data availability prior to invoking .readline.

Traditional Blocking Approach

Typically, reads on standard output are blocking, meaning the execution halts until data becomes available. This is demonstrated below:

import subprocess

p = subprocess.Popen('myprogram.exe', stdout = subprocess.PIPE)
output_str = p.stdout.readline()

However, in this approach, the execution will stall if no data is immediately present in the stream.

Overcoming Blocking Reads with Queue.get_nowait()

To circumvent blocking reads, a reliable cross-platform approach is to employ the Queue module and its get_nowait() method. This method gracefully handles the absence of data in the stream, allowing for non-blocking reads:

import sys
from subprocess import PIPE, Popen
from threading  import Thread
from queue import Queue, Empty

ON_POSIX = 'posix' in sys.builtin_module_names

def enqueue_output(out, queue):
    for line in iter(out.readline, b''):
        queue.put(line)
    out.close()

p = Popen(['myprogram.exe'], stdout=PIPE, bufsize=1, close_fds=ON_POSIX)
q = Queue()
t = Thread(target=enqueue_output, args=(p.stdout, q))
t.daemon = True # thread dies with the program
t.start()

try:  line = q.get_nowait()
except Empty:
    print('no output yet')
else: # got line
    # ... do something with line

In this approach, a separate thread is spawned to continuously enqueue output from the subprocess's standard output into the queue. The main thread can then perform non-blocking reads by calling get_nowait(). If the queue is empty, the call returns without blocking, allowing the main thread to proceed.

The above is the detailed content of How Can I Achieve Non-Blocking Reads from a Subprocess's Standard Output?. For more information, please follow other related articles on the PHP Chinese website!