可以将文章内容翻译成中文,广告屏蔽插件可能会导致该功能失效(如失效,请关闭广告屏蔽插件后再试):
问题:
I'm looking for a Python solution that will allow me to save the output of a command in a file without hiding it from the console.
FYI: I'm asking about tee (as the Unix command line utility) and not the function with the same name from Python intertools module.
Details
- Python solution (not calling
tee
, it is not available under Windows)
- I do not need to provide any input to stdin for called process
- I have no control over the called program. All I know is that it will output something to stdout and stderr and return with an exit code.
- To work when calling external programs (subprocess)
- To work for both
stderr
and stdout
- Being able to differentiate between stdout and stderr because I may want to display only one of the to the console or I could try to output stderr using a different color - this means that
stderr = subprocess.STDOUT
will not work.
- Live output (progressive) - the process can run for a long time, and I'm not able to wait for it to finish.
- Python 3 compatible code (important)
References
Here are some incomplete solutions I found so far:
- http://devlishgenius.blogspot.com/2008/10/logging-in-real-time-in-python.html (mkfifo works only on Unix)
- http://blog.kagesenshi.org/2008/02/teeing-python-subprocesspopen-output.html (doesn't work at all)
Diagram http://blog.i18n.ro/wp-content/uploads/2010/06/Drawing_tee_py.png
Current code (second try)
#!/usr/bin/python
from __future__ import print_function
import sys, os, time, subprocess, io, threading
cmd = "python -E test_output.py"
from threading import Thread
class StreamThread ( Thread ):
def __init__(self, buffer):
Thread.__init__(self)
self.buffer = buffer
def run ( self ):
while 1:
line = self.buffer.readline()
print(line,end="")
sys.stdout.flush()
if line == '':
break
proc = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdoutThread = StreamThread(io.TextIOWrapper(proc.stdout))
stderrThread = StreamThread(io.TextIOWrapper(proc.stderr))
stdoutThread.start()
stderrThread.start()
proc.communicate()
stdoutThread.join()
stderrThread.join()
print("--done--")
#### test_output.py ####
#!/usr/bin/python
from __future__ import print_function
import sys, os, time
for i in range(0, 10):
if i%2:
print("stderr %s" % i, file=sys.stderr)
else:
print("stdout %s" % i, file=sys.stdout)
time.sleep(0.1)
Real output
stderr 1
stdout 0
stderr 3
stdout 2
stderr 5
stdout 4
stderr 7
stdout 6
stderr 9
stdout 8
--done--
Expected output was to have the lines ordered. Remark, modifying the Popen to use only one PIPE is not allowed because in the real life I will want to do different things with stderr and stdout.
Also even in the second case I was not able to obtain real-time like out, in fact all the results were received when the process finished. By default, Popen should use no buffers (bufsize=0).
回答1:
I see that this is a rather old post but just in case someone is still searching for a way to do this:
proc = subprocess.Popen(["ping", "localhost"],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
with open("logfile.txt", "w") as log_file:
while proc.poll() is None:
line = proc.stderr.readline()
if line:
print "err: " + line.strip()
log_file.write(line)
line = proc.stdout.readline()
if line:
print "out: " + line.strip()
log_file.write(line)
回答2:
Finally I had to implement tee()
command in Python myself.
You can get it from here http://github.com/pycontribs/tendo/blob/master/tendo/tee.py
Currently it does allow you to do things like:
tee("python --v") # works just like os.system()
tee("python --v", "log.txt") # file names
tee("python --v", file_handle)
import logging
tee("python --v", logging.info) # receive a method
The only current limitation is that it is not able to differentiate between stderr
and stdout
, meaning that it will merge both of them.
回答3:
This is a straightforward port of tee
to Python.
import sys
sinks = sys.argv[1:]
sinks = [open(sink, "w") for sink in sinks]
sinks.append(sys.stderr)
while True:
input = sys.stdin.read(1024)
if input:
for sink in sinks:
sink.write(input)
else:
break
I'm running on Linux right now but this ought to work on most platforms.
Now for the subprocess
part, I don't know how you want to 'wire' the subprocess's stdin
, stdout
and stderr
to your stdin
, stdout
, stderr
and file sinks, but I know you can do this:
import subprocess
callee = subprocess.Popen( ["python", "-i"],
stdin = subprocess.PIPE,
stdout = subprocess.PIPE,
stderr = subprocess.PIPE
)
Now you can access callee.stdin
, callee.stdout
and callee.stderr
like normal files, enabling the above "solution" to work. If you want to get the callee.returncode
, you'll need to make an extra call to callee.poll()
.
Be careful with writing to callee.stdin
: if the process has exited when you do that, an error may be rised (on Linux, I get IOError: [Errno 32] Broken pipe
).
回答4:
If you don't want to interact with the process you can use the subprocess module just fine.
Example:
tester.py
import os
import sys
for file in os.listdir('.'):
print file
sys.stderr.write("Oh noes, a shrubbery!")
sys.stderr.flush()
sys.stderr.close()
testing.py
import subprocess
p = subprocess.Popen(['python', 'tester.py'], stdout=subprocess.PIPE,
stdin=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = p.communicate()
print stdout, stderr
In your situation you can simply write stdout/stderr to a file first. You can send arguments to your process with communicate as well, though I wasn't able to figure out how to continually interact with the subprocess.
回答5:
There are subtle problems/bugs in python related to subprocess.PIPE:
http://bugs.python.org/issue1652
Apparently this was fixed in python3+, but not in python 2.7 and older. For that you need to use:
code.google.com/p/python-subprocess32/
回答6:
Try this :
import sys
class tee-function :
def __init__(self, _var1, _var2) :
self.var1 = _var1
self.var2 = _var2
def __del__(self) :
if self.var1 != sys.stdout and self.var1 != sys.stderr :
self.var1.close()
if self.var2 != sys.stdout and self.var2 != sys.stderr :
self.var2.close()
def write(self, text) :
self.var1.write(text)
self.var2.write(text)
def flush(self) :
self.var1.flush()
self.var2.flush()
stderrsav = sys.stderr
out = open(log, "w")
sys.stderr = tee-function(stderrsav, out)
回答7:
I wrote a thing that wraps shell commands in Python.
Key advantages:
- This util captures stdout/stderr always
- This util provides an option to echo stdout/stderr to stdout/stderr for the process
- When echoing stdout/stderr the out/err there is no delay
Key disadvantage:
source: https://gist.github.com/AndrewHoos/9f03c74988469b517a7a