How Can I Simultaneously Capture and Log Live Output from a Subprocess in Python?

Live Output from Subprocess Command

When utilizing subprocess.Popen to execute external commands, it becomes challenging to obtain both live output streaming and output storage for logging and error checking. This article explores techniques to address this issue, providing solutions that simultaneously capture and display output in real time while maintaining the ability to write it to a log file.

Solution 1: Iterator-Based Approach

One method involves creating an iterator from the read function and writing the output to both the standard output and a log file:

import subprocess
import sys

with open("log.txt", "wb") as f:
    process = subprocess.Popen(your_command, stdout=subprocess.PIPE)
    for c in iter(lambda: process.stdout.read(1), b""):
        sys.stdout.buffer.write(c)
        f.buffer.write(c)

In Python 3, replace "w" with "wb" to support binary writing.

Solution 2: Reader and Writer File Approach

Alternatively, a reader and writer file can be employed to separate stdout and file writing:

import io
import subprocess
import sys

filename = "log.txt"
with io.open(filename, "wb") as writer, io.open(filename, "rb", 1) as reader:
    process = subprocess.Popen(command, stdout=writer)
    while process.poll() is None:
        sys.stdout.write(reader.read())
        time.sleep(0.5)
    # Read the remaining
    sys.stdout.write(reader.read())

This method ensures that data is written to both the standard output and the log file, while offering the advantage of asynchronous reading and writing.

The above is the detailed content of How Can I Simultaneously Capture and Log Live Output from a Subprocess in Python?. For more information, please follow other related articles on the PHP Chinese website!