Get output of pv using python

问题:

Is there a way to use the program pv from within python so as to get the progress of an operation?

So far I have the following:

    p0 = sp.Popen(["pv", "-f", args.filepath],
                  bufsize=0,
                  stdout=sp.PIPE,
                  stderr=sp.PIPE)
    p1 = sp.Popen(["awk", "{print $1, $2, $1, $3, $4 }",  "{}".format(args.filepath)],
                   stdout=sp.PIPE,
                   stdin=p0.stdout)

But I'm having trouble getting continuous output from p0. I tried:

    for line in p0.stderr:
        print("line:", line)

But this waits for the process to finish and then only prints the last progress report from pv. Does anybody know how I can get it to print the continuously updating status?

回答1:

It turns out pv outputs each line with a carriage return at end (\r). To be able to continuously read from the output, Popen needs to be initialized with universal_lines=True, like this:

    p0 = sp.Popen(['pv', '-f', args.filepath],
                    stdout=sp.PIPE,
                    stderr=sp.PIPE,
                    universal_newlines=True)

This leads to a continuous output of progress reports:

line: 7.12MB 0:00:01 [ 7.1MB/s] [=>                                  ]  8% ETA 0:00:11

line: 14.6MB 0:00:02 [7.42MB/s] [====>                               ] 16% ETA 0:00:10

line: 22.1MB 0:00:03 [7.55MB/s] [=======>                            ] 24% ETA 0:00:09

line: 29.5MB 0:00:04 [7.36MB/s] [==========>                         ] 33% ETA 0:00:08

Here's a reference to a similar question:

Real time output of subprocess.popen() and not line by line

回答2:

try reading from p0.stderr

pv leaves stdout untouched, it writes only to stderr