How Can I Stream Subprocess Output in Real-Time Using Python?

Streaming Output from Subprocess.communicate()

When using subprocess.communicate() to retrieve output from a process, the entirety of the standard output (stdout) may be returned at once, preventing real-time monitoring of the output. This issue arises because of the blocking nature of the communicate() function.

To address this, one can adopt a streaming approach to printing each line of stdout as it becomes available. This can be achieved by employing iterators and manipulating the bufsize parameter of Popen().

In Python 2:

from subprocess import Popen, PIPE

p = Popen(["cmd", "arg1"], stdout=PIPE, bufsize=1)
with p.stdout:
    for line in iter(p.stdout.readline, b''):
        print(line)
p.wait()

In Python 3, the code becomes:

from subprocess import Popen, PIPE

with Popen(["cmd", "arg1"], stdout=PIPE, bufsize=1, universal_newlines=True) as p:
    for line in p.stdout:
        print(line, end='')

The bufsize=1 parameter forces the process to flush its output buffer after each line, ensuring that each line is printed immediately. iter() allows lines to be read asynchronously, making it possible to stream the output. The process termination is handled with the wait() method.

By using this streaming approach, you can monitor the output of a long-running process in real time without compromising the synchronous nature of the communicate() function.

The above is the detailed content of How Can I Stream Subprocess Output in Real-Time Using Python?. For more information, please follow other related articles on the PHP Chinese website!