Python - capture Popen stdout AND display on conso

2019-01-24 16:50发布

I want to capture stdout from a long-ish running process started via subprocess.Popen(...) so I'm using stdout=PIPE as an arg.

However, because it's a long running process I also want to send the output to the console (as if I hadn't piped it) to give the user of the script an idea that it's still working.

Is this at all possible?

Cheers.

5条回答
家丑人穷心不美
2楼-- · 2019-01-24 16:54

S. Lott's comment points to Getting realtime output using subprocess and Real-time intercepting of stdout from another process in Python

I'm curious that Alex's answer here is different from his answer 1085071. My simple little experiments with the answers in the two other referenced questions has given good results...

I went and looked at wexpect as per Alex's answer above, but I have to say reading the comments in the code I was not left a very good feeling about using it.

I guess the meta-question here is when will pexpect/wexpect be one of the Included Batteries?

查看更多
乱世女痞
3楼-- · 2019-01-24 17:06

The buffering your long-running sub-process is probably performing will make your console output jerky and very bad UX. I suggest you consider instead using pexpect (or, on Windows, wexpect) to defeat such buffering and get smooth, regular output from the sub-process. For example (on just about any unix-y system, after installing pexpect):

>>> import pexpect
>>> child = pexpect.spawn('/bin/bash -c "echo ba; sleep 1; echo bu"', logfile=sys.stdout); x=child.expect(pexpect.EOF); child.close()
ba
bu
>>> child.before
'ba\r\nbu\r\n'

The ba and bu will come with the proper timing (about a second between them). Note the output is not subject to normal terminal processing, so the carriage returns are left in there -- you'll need to post-process the string yourself (just a simple .replace!-) if you need \n as end-of-line markers (the lack of processing is important just in case the sub-process is writing binary data to its stdout -- this ensures all the data's left intact!-).

查看更多
我想做一个坏孩纸
4楼-- · 2019-01-24 17:09

Can you simply print it as you read it from the pipe?

查看更多
Lonely孤独者°
5楼-- · 2019-01-24 17:09

Inspired by pty.openpty() suggestion somewhere above, tested on python2.6, linux. Publishing since it took a while to make this working properly, w/o buffering...

def call_and_peek_output(cmd, shell=False):
    import pty, subprocess
    master, slave = pty.openpty()
    p = subprocess.Popen(cmd, shell=shell, stdin=None, stdout=slave, close_fds=True)
    os.close(slave)
    line = ""
    while True:
        try:
            ch = os.read(master, 1)
        except OSError:
            # We get this exception when the spawn process closes all references to the
            # pty descriptor which we passed him to use for stdout
            # (typically when it and its childs exit)
            break
        line += ch
        sys.stdout.write(ch)
        if ch == '\n':
            yield line
            line = ""
    if line:
        yield line

    ret = p.wait()
    if ret:
        raise subprocess.CalledProcessError(ret, cmd)

for l in call_and_peek_output("ls /", shell=True):
    pass
查看更多
We Are One
6楼-- · 2019-01-24 17:09

Alternatively, you can pipe your process into tee and capture only one of the streams. Something along the lines of sh -c 'process interesting stuff' | tee /dev/stderr.

Of course, this only works on Unix-like systems.

查看更多
登录 后发表回答