Home >Backend Development >Python Tutorial >Python standard output redirection method
Python standard output redirection method
- 高洛峰Original
- 2017-02-25 13:11:213070browse
1. Background
In Python, the file objects sys.stdin, sys.stdout and sys.stderr corresponds to the standard input, standard output and standard error streams of the interpreter respectively. When the program starts, the initial values of these objects are saved by sys.__stdin__, sys.__stdout__ and sys.__stderr__ for finalization. Restore standard stream objects.
IDLE (Python GUI) in Windows system is run by pythonw.exe, and the GUI does not have a console. Therefore, IDLE replaces the standard output handle with a special PseudoOutputFile object so that script output is redirected to the IDLE terminal window (Shell). This can lead to some strange problems, for example:
Python 2.7.11 (v2.7.11:6d1b6a68f775, Dec 5 2015, 20:32:19) [MSC v.1500 32 bit (Intel)] on win32 Type "copyright", "credits" or "license()" for more information. >>> import sys >>> for fd in (sys.stdin, sys.stdout, sys.stderr): print fd <idlelib.PyShell.PseudoInputFile object at 0x0177C910> <idlelib.PyShell.PseudoOutputFile object at 0x0177C970> <idlelib.PyShell.PseudoOutputFile object at 0x017852B0> >>> for fd in (sys.__stdin__, sys.__stdout__, sys.__stderr__): print fd <open file '<stdin>', mode 'r' at 0x00FED020> <open file '<stdout>', mode 'w' at 0x00FED078> <open file '<stderr>', mode 'w' at 0x00FED0D0> >>>
can be found that sys.__stdout__ is different from sys.stdoutThe values are not the same. When running the above code under a normal Python interpreter (such as through a Windows console), the two values are the same.
When the print statement does not end with a comma, a newline character (linefeed) will be automatically appended to the end of the output string; otherwise, a space will be used to replace the additional newline character. The print statement writes to the standard output stream by default, and can also be redirected to a file or other writable object (all objects that provide a write method). In this way, you can use a concise print statement instead of the clumsy object.write('hello'+'\n') writing method.
It can be seen from the above that when calling print obj to print an object in Python, by default it is equivalent to calling sys.stdout.write(obj+'\n')
Examples are as follows:
>>> import sys >>> print 'Hello World' Hello World >>> sys.stdout.write('Hello World') Hello World
2. Redirection method
This section introduces commonly used Python standard output redirection methods. Each of these methods has its own advantages and disadvantages and is suitable for different scenarios.
2.1 Console Redirection
The simplest and most commonly used way to redirect output is to use console commands. This redirection is done by the console and has nothing to do with Python itself.
Windows command prompt (cmd.exe) and Linux Shell (bash, etc.) both redirect output through ">" or ">>". Among them, ">" means overwriting content, and ">>" means appending content. Similarly, "2>" redirects standard error. Redirecting to "nul" (Windows) or "/dev/null" (Linux) suppresses output and neither displays nor saves to disk.
Taking the Windows command prompt as an example, redirect the Python script output to a file (blank lines between commands have been deleted to shorten the length):
E:\>echo print 'hello' > test.py E:\>test.py > out.txt E:\>type out.txt hello E:\>test.py >> out.txt E:\>type out.txt hello hello E:\>test.py > nul
Note, when executing a Python script in the Windows command prompt, the command line does not need to start with "python", the system will automatically call the Python interpreter based on the script suffix. In addition, the type command can directly display the contents of a text file, similar to the cat command in Linux systems.
When executing a Python script in the Linux Shell, the command line should start with "python". In addition to ">" or ">>" redirection, the tee command can also be used. This command can output the content to the terminal screen and (multiple) files at the same time. The "-a" option means append writing, otherwise it will overwrite the writing. The example is as follows (echo $SHELL or echo $0 displays the currently used Shell):
[wangxiaoyuan_@localhost ~]$ echo $SHELL /bin/bash [wangxiaoyuan_@localhost ~]$ python -c "print 'hello'" hello [wangxiaoyuan_@localhost ~]$ python -c "print 'hello'" > out.txt [wangxiaoyuan_@localhost ~]$ cat out.txt hello [wangxiaoyuan_@localhost ~]$ python -c "print 'world'" >> out.txt [wangxiaoyuan_@localhost ~]$ cat out.txt hello world [wangxiaoyuan_@localhost ~]$ python -c "print 'I am'" | tee out.txt I am [wangxiaoyuan_@localhost ~]$ python -c "print 'xywang'" | tee -a out.txt xywang [wangxiaoyuan_@localhost ~]$ cat out.txt I am xywang [wangxiaoyuan_@localhost ~]$ python -c "print 'hello'" > /dev/null [wangxiaoyuan_@localhost ~]$
If only If you want to save the script output to a file, you can also directly use the log capture function of the session window.
Note that the impact of console redirection is global and only applies to relatively simple output tasks.
2.2 print >> redirection
This method is based on the expanded form of the print statement, that is, "print obj >> expr". Among them, obj is a file-like (especially one that provides the write method) object, and when it is None, it corresponds to the standard output (sys.stdout). expr will be output to the file object.
The example is as follows:
memo = cStringIO.StringIO(); serr = sys.stderr; file = open('out.txt', 'w+') print >>memo, 'StringIO'; print >>serr, 'stderr'; print >>file, 'file' print >>None, memo.getvalue()
After the above code is executed, the screen will display "serr" and "StringIO" (two lines, pay attention to the order) , write "file" in the out.txt file.
It can be seen that this method is very flexible and convenient. The disadvantage is that it is not suitable for scenarios with many output statements.
2.3 sys.stdout redirection
Assign a writable object (such as a file-like object) to sys.stdout to enable subsequent print Statements are output to this object. After the redirection is completed, sys.stdout should be restored to the original default value, which is standard output.
A simple example is as follows:
import sys savedStdout = sys.stdout #保存标准输出流 with open('out.txt', 'w+') as file: sys.stdout = file #标准输出重定向至文件 print 'This message is for file!' sys.stdout = savedStdout #恢复标准输出流 print 'This message is for screen!'
Note that the initial value of sys.stdout in IDLE is the PseudoOutputFile object, which is the same as sys.__stdout__ is not the same. For the sake of generality, this example defines a separate variable (savedStdout) to save sys.stdout, which will also be done below. Additionally, this example does not apply to stdout objects imported via from sys import stdout.
The following will customize a variety of file-like objects with write() methods to meet different needs:
class RedirectStdout: #import os, sys, cStringIO
def __init__(self):
self.content = ''
self.savedStdout = sys.stdout
self.memObj, self.fileObj, self.nulObj = None, None, None
#外部的print语句将执行本write()方法，并由当前sys.stdout输出
def write(self, outStr):
#self.content.append(outStr)
self.content += outStr
def toCons(self): #标准输出重定向至控制台
sys.stdout = self.savedStdout #sys.__stdout__
def toMemo(self): #标准输出重定向至内存
self.memObj = cStringIO.StringIO()
sys.stdout = self.memObj
def toFile(self, file='out.txt'): #标准输出重定向至文件
self.fileObj = open(file, 'a+', 1) #改为行缓冲
sys.stdout = self.fileObj
def toMute(self): #抑制输出
self.nulObj = open(os.devnull, 'w')
sys.stdout = self.nulObj
def restore(self):
self.content = ''
if self.memObj.closed != True:
self.memObj.close()
if self.fileObj.closed != True:
self.fileObj.close()
if self.nulObj.closed != True:
self.nulObj.close()
sys.stdout = self.savedStdout #sys.__stdout__
注意，toFile()方法中，open(name[, mode[, buffering]])调用选择行缓冲(无缓冲会影响性能)。这是为了观察中间写入过程，否则只有调用close()或flush()后输出才会写入文件。内部调用open()方法的缺点是不便于用户定制写文件规则，如模式(覆盖或追加)和缓冲(行或全缓冲)。
重定向效果如下：
redirObj = RedirectStdout() sys.stdout = redirObj #本句会抑制"Let's begin!"输出 print "Let's begin!" #屏显'Hello World!'和'I am xywang.'(两行) redirObj.toCons(); print 'Hello World!'; print 'I am xywang.' #写入'How are you?'和"Can't complain."(两行) redirObj.toFile(); print 'How are you?'; print "Can't complain." redirObj.toCons(); print "What'up?" #屏显 redirObj.toMute(); print '<Silence>' #无屏显或写入 os.system('echo Never redirect me!') #控制台屏显'Never redirect me!' redirObj.toMemo(); print 'What a pity!' #无屏显或写入 redirObj.toCons(); print 'Hello?' #屏显 redirObj.toFile(); print "Oh, xywang can't hear me" #该串写入文件 redirObj.restore() print 'Pop up' #屏显
可见，执行toXXXX()语句后，标准输出流将被重定向到XXXX。此外，toMute()和toMemo()的效果类似，均可抑制输出。
使用某对象替换sys.stdout时，尽量确保该对象接近文件对象，尤其是涉及第三方库时(该库可能使用sys.stdout的其他方法)。此外，本节替换sys.stdout的代码实现并不影响由os.popen()、os.system()或os.exec*()系列方法所创建进程的标准I/O流。
2.4 上下文管理器(Context Manager)
本节严格意义上并非新的重定向方式，而是利用Pyhton上下文管理器优化上节的代码实现。借助于上下文管理器语法，可不必向重定向使用者暴露sys.stdout。
首先考虑输出抑制，基于上下文管理器语法实现如下：
import sys, cStringIO, contextlib
class DummyFile:
def write(self, outStr): pass
@contextlib.contextmanager
def MuteStdout():
savedStdout = sys.stdout
sys.stdout = cStringIO.StringIO() #DummyFile()
try:
yield
except Exception: #捕获到错误时，屏显被抑制的输出(该处理并非必需)
content, sys.stdout = sys.stdout, savedStdout
print content.getvalue()#; raise
#finally:
sys.stdout = savedStdout
使用示例如下：
with MuteStdout(): print "I'll show up when <raise> is executed!" #不屏显不写入 raise #屏显上句 print "I'm hiding myself somewhere:)" #不屏显
再考虑更通用的输出重定向：
import os, sys
from contextlib import contextmanager
@contextmanager
def RedirectStdout(newStdout):
savedStdout, sys.stdout = sys.stdout, newStdout
try:
yield
finally:
sys.stdout = savedStdout
使用示例如下：
def Greeting(): print 'Hello, boss!'
with open('out.txt', "w+") as file:
print "I'm writing to you..." #屏显
with RedirectStdout(file):
print 'I hope this letter finds you well!' #写入文件
print 'Check your mailbox.' #屏显
with open(os.devnull, "w+") as file, RedirectStdout(file):
Greeting() #不屏显不写入
print 'I deserve a pay raise:)' #不屏显不写入
print 'Did you hear what I said?' #屏显
可见，with内嵌块里的函数和print语句输出均被重定向。注意，上述示例不是线程安全的，主要适用于单线程。
当函数被频繁调用时，建议使用装饰器包装该函数。这样，仅需修改该函数定义，而无需在每次调用该函数时使用with语句包裹。示例如下：
import sys, cStringIO, functools
def MuteStdout(retCache=False):
def decorator(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
savedStdout = sys.stdout
sys.stdout = cStringIO.StringIO()
try:
ret = func(*args, **kwargs)
if retCache == True:
ret = sys.stdout.getvalue().strip()
finally:
sys.stdout = savedStdout
return ret
return wrapper
return decorator
若装饰器MuteStdout的参数retCache为真，外部调用func()函数时将返回该函数内部print输出的内容(可供屏显)；若retCache为假，外部调用func()函数时将返回该函数的返回值(抑制输出)。
MuteStdout装饰器使用示例如下：
@MuteStdout(True) def Exclaim(): print 'I am proud of myself!' @MuteStdout() def Mumble(): print 'I lack confidence...'; return 'sad' print Exclaim(), Exclaim.__name__ #屏显'I am proud of myself! Exclaim' print Mumble(), Mumble.__name__ #屏显'sad Mumble'
在所有线程中，被装饰函数执行期间，sys.stdout都会被MuteStdout装饰器劫持。而且，函数一经装饰便无法移除装饰。因此，使用该装饰器时应慎重考虑场景。
接着，考虑创建RedirectStdout装饰器：
def RedirectStdout(newStdout=sys.stdout):
def decorator(func):
def wrapper(*args,**kwargs):
savedStdout, sys.stdout = sys.stdout, newStdout
try:
return func(*args, **kwargs)
finally:
sys.stdout = savedStdout
return wrapper
return decorator
使用示例如下：
file = open('out.txt', "w+") @RedirectStdout(file) def FunNoArg(): print 'No argument.' @RedirectStdout(file) def FunOneArg(a): print 'One argument:', a def FunTwoArg(a, b): print 'Two arguments: %s, %s' %(a,b) FunNoArg() #写文件'No argument.' FunOneArg(1984) #写文件'One argument: 1984' RedirectStdout()(FunTwoArg)(10,29) #屏显'Two arguments: 10, 29' print FunNoArg.__name__ #屏显'wrapper'(应显示'FunNoArg') file.close()
注意FunTwoArg()函数的定义和调用与其他函数的不同，这是两种等效的语法。此外，RedirectStdout装饰器的最内层函数wrapper()未使用"functools.wraps(func)"修饰，会丢失被装饰函数原有的特殊属性(如函数名、文档字符串等)。
2.5 logging模块重定向
对于代码量较大的工程，建议使用logging模块进行输出。该模块是线程安全的，可将日志信息输出到控制台、写入文件、使用TCP/UDP协议发送到网络等等。
默认情况下logging模块将日志输出到控制台(标准出错)，且只显示大于或等于设置的日志级别的日志。日志级别由高到低为CRITICAL > ERROR > WARNING > INFO > DEBUG > NOTSET，默认级别为WARNING。
以下示例将日志信息分别输出到控制台和写入文件：
import logging logging.basicConfig(level = logging.DEBUG, format = '%(asctime)s [%(levelname)s] at %(filename)s,%(lineno)d: %(message)s', datefmt = '%Y-%m-%d(%a)%H:%M:%S', filename = 'out.txt', filemode = 'w') #将大于或等于INFO级别的日志信息输出到StreamHandler(默认为标准错误) console = logging.StreamHandler() console.setLevel(logging.INFO) formatter = logging.Formatter('[%(levelname)-8s] %(message)s') #屏显实时查看，无需时间 console.setFormatter(formatter) logging.getLogger().addHandler(console) logging.debug('gubed'); logging.info('ofni'); logging.critical('lacitirc')
通过对多个handler设置不同的level参数，可将不同的日志内容输入到不同的地方。本例使用在logging模块内置的StreamHandler(和FileHandler)，运行后屏幕上显示:
[INFO ] ofni [CRITICAL] lacitirc
out.txt文件内容则为:
2016-05-13(Fri)17:10:53 [DEBUG] at test.py,25: gubed 2016-05-13(Fri)17:10:53 [INFO] at test.py,25: ofni 2016-05-13(Fri)17:10:53 [CRITICAL] at test.py,25: lacitirc
除直接在程序中设置Logger、Handler、Formatter等外，还可将这些信息写入配置文件。示例如下：
#logger.conf ###############Logger############### [loggers] keys=root,Logger2F,Logger2CF [logger_root] level=DEBUG handlers=hWholeConsole [logger_Logger2F] handlers=hWholeFile qualname=Logger2F propagate=0 [logger_Logger2CF] handlers=hPartialConsole,hPartialFile qualname=Logger2CF propagate=0 ###############Handler############### [handlers] keys=hWholeConsole,hPartialConsole,hWholeFile,hPartialFile [handler_hWholeConsole] class=StreamHandler level=DEBUG formatter=simpFormatter args=(sys.stdout,) [handler_hPartialConsole] class=StreamHandler level=INFO formatter=simpFormatter args=(sys.stderr,) [handler_hWholeFile] class=FileHandler level=DEBUG formatter=timeFormatter args=('out.txt', 'a') [handler_hPartialFile] class=FileHandler level=WARNING formatter=timeFormatter args=('out.txt', 'w') ###############Formatter############### [formatters] keys=simpFormatter,timeFormatter [formatter_simpFormatter] format=[%(levelname)s] at %(filename)s,%(lineno)d: %(message)s [formatter_timeFormatter] format=%(asctime)s [%(levelname)s] at %(filename)s,%(lineno)d: %(message)s datefmt=%Y-%m-%d(%a)%H:%M:%S
此处共创建三个Logger：root，将所有日志输出至控制台；Logger2F，将所有日志写入文件；Logger2CF，将级别大于或等于INFO的日志输出至控制台，将级别大于或等于WARNING的日志写入文件。
程序以如下方式解析配置文件和重定向输出：
import logging, logging.config
logging.config.fileConfig("logger.conf")
logger = logging.getLogger("Logger2CF")
logger.debug('gubed'); logger.info('ofni'); logger.warn('nraw')
logger.error('rorre'); logger.critical('lacitirc')
logger1 = logging.getLogger("Logger2F")
logger1.debug('GUBED'); logger1.critical('LACITIRC')
logger2 = logging.getLogger()
logger2.debug('gUbEd'); logger2.critical('lAcItIrC')
运行后屏幕上显示:
[INFO] at test.py,7: ofni [WARNING] at test.py,7: nraw [ERROR] at test.py,8: rorre [CRITICAL] at test.py,8: lacitirc [DEBUG] at test.py,14: gUbEd [CRITICAL] at test.py,14: lAcItIrC
out.txt文件内容则为:
2016-05-13(Fri)20:31:21 [WARNING] at test.py,7: nraw 2016-05-13(Fri)20:31:21 [ERROR] at test.py,8: rorre 2016-05-13(Fri)20:31:21 [CRITICAL] at test.py,8: lacitirc 2016-05-13(Fri)20:31:21 [DEBUG] at test.py,11: GUBED 2016-05-13(Fri)20:31:21 [CRITICAL] at test.py,11: LACITIRC
三. 总结
以上就是关于Python标准输出的重定向方式的全部内容，希望对学习python的朋友们能有所帮助，如果有疑问欢迎大家留言讨论。
For more articles related to python standard output redirection methods, please pay attention to the PHP Chinese website!