How to Avoid Python Programs from Hanging When Reading Process Output?

Stop reading process output in Python without hang?

Problem:

A Python program needs to interact with an external process (e.g., "top") that continuously produces output. However, simply reading the output directly can cause the program to hang indefinitely.

Solution:

To prevent hanging, it's essential to employ non-blocking or asynchronous mechanisms when reading process output. Here are a few possible approaches:

Spooled Temporary File (Recommended)

This method utilizes a dedicated file object to store the process output.

#!/usr/bin/env python<br>import subprocess<br>import tempfile<br>import time</p>
<p>def main():</p>
<pre class="brush:php;toolbar:false"># Open a temporary file (automatically deleted on closure)
f = tempfile.TemporaryFile()

# Start the process and redirect stdout to the file
p = subprocess.Popen(["top"], stdout=f)

# Wait for a specified duration
time.sleep(2)

# Kill the process
p.terminate()
p.wait()

# Rewind and read the captured output from the file
f.seek(0)
output = f.read()

# Print the output
print(output)
f.close()

if name == "__main__":

main()

Thread-based Output Reading

This approach employs a separate thread to continuously read the process output while the main thread proceeds with other tasks.

import collections<br>import subprocess<br>import threading<br>import time</p>
<p>def read_output(process, append):</p>
<pre class="brush:php;toolbar:false">for line in iter(process.stdout.readline, ""):
    append(line)

def main():

# Start the process and redirect stdout
process = subprocess.Popen(["top"], stdout=subprocess.PIPE, close_fds=True)

# Create a thread for output reading
q = collections.deque(maxlen=200)
t = threading.Thread(target=read_output, args=(process, q.append))
t.daemon = True
t.start()

# Wait for the specified duration
time.sleep(2)

# Print the saved output
print(''.join(q))

if name == "__main__":

main()

signal.alarm() (Unix-only)

This method uses Unix signals to terminate the process after a specified timeout, regardless of whether all output has been read.

import collections<br>import signal<br>import subprocess</p>
<p>class Alarm(Exception):</p>
<pre class="brush:php;toolbar:false">pass

def alarm_handler(signum, frame):

raise Alarm

def main():

# Start the process and redirect stdout
process = subprocess.Popen(["top"], stdout=subprocess.PIPE, close_fds=True)

# Set signal handler
signal.signal(signal.SIGALRM, alarm_handler)
signal.alarm(2)

try:
    # Read and save a specified number of lines
    q = collections.deque(maxlen=200)
    for line in iter(process.stdout.readline, ""):
        q.append(line)
    signal.alarm(0)  # Cancel alarm
except Alarm:
    process.terminate()
finally:
    # Print the saved output
    print(''.join(q))

if name == "__main__":

main()

threading.Timer

This approach employs a timer to terminate the process after a specified timeout. It works on both Unix and Windows systems.

import collections<br>import subprocess<br>import threading</p>
<p>def main():</p>
<pre class="brush:php;toolbar:false"># Start the process and redirect stdout
process = subprocess.Popen(["top"], stdout=subprocess.PIPE, close_fds=True)

# Create a timer for process termination
timer = threading.Timer(2, process.terminate)
timer.start()

# Read and save a specified number of lines
q = collections.deque(maxlen=200)
for line in iter(process.stdout.readline, ""):
    q.append(line)
timer.cancel()

# Print the saved output
print(''.join(q))

if name == "__main__":

main()

No Threads, No Signals

This method uses a simple time-based loop to check for process output and kill it if it exceeds a specified timeout.

import collections<br>import subprocess<br>import sys<br>import time</p>
<p>def main():</p>
<pre class="brush:php;toolbar:false">args = sys.argv[1:]
if not args:
    args = ['top']

# Start the process and redirect stdout
process = subprocess.Popen(args, stdout=subprocess.PIPE, close_fds=True)

# Save a specified number of lines
q = collections.deque(maxlen=200)

# Set a timeout duration
timeout = 2

now = start = time.time()
while (now - start) < timeout:
    line = process.stdout.readline()
    if not line:
        break
    q.append(line)
    now = time.time()
else:  # On timeout
    process.terminate()

# Print the saved output
print(''.join(q))

if name == "__main__":

main()

Note: The number of lines stored can be adjusted as needed by setting the 'maxlen' parameter of the deque data structure.

The above is the detailed content of How to Avoid Python Programs from Hanging When Reading Process Output?. For more information, please follow other related articles on the PHP Chinese website!