Merge child process stdout and stderr

问题:

How do I merge child process stdout and stderr?

The following does not work since ownership cannot be shared between stdout and stderr:

let pipe = Stdio::piped();
let prog = Command::new("prog")
                        .stdout(pipe)
                        .stderr(pipe)
                        .spawn()
                        .expect("failed to execute prog");

In other words, what is the Rust equivalent of 2>&1 in the shell?

回答1:

I'm working on a library called duct that makes this easy. It's not totally stable yet, and not documented (though it maps very closely to the Python version), but it works today:

#[macro_use]
extern crate duct;

fn main() {
    cmd!("echo", "hi").stderr_to_stdout().run();
}

The "right way" to do something like this, which duct is doing for you under the covers, is to create a double-ended OS pipe and pass the write end of it to both stdout and stderr. The standard library's Command class supports this sort of thing in general because Stdio implements FromRawFd, but unfortunately the standard library doesn't expose a way to create pipes. I've written another library called os_pipe to do this inside of duct, and if you want you can use it directly.

This has been tested on Linux, Windows, and macOS.

回答2:

I see nothing in the standard library that does this for you. Doesn't mean you can't write it yourself. This also means you get to decide how frequently each file descriptor is read and how to combine the data from each of the file descriptors. Here, I attempt to read in chunks using the default BufReader size and prefer to put stdout data first when both descriptors have data.

use std::io::prelude::*;
use std::io::BufReader;
use std::process::{Command, Stdio};

fn main() {
    let mut child =
        Command::new("/tmp/output")
        .stdout(Stdio::piped())
        .stderr(Stdio::piped())
        .spawn()
        .expect("Couldn't run program");

    let mut output = Vec::new();

    // Should be moved to a function that accepts something implementing `Write`
    {
        let stdout = child.stdout.as_mut().expect("Wasn't stdout");
        let stderr = child.stderr.as_mut().expect("Wasn't stderr");

        let mut stdout = BufReader::new(stdout);
        let mut stderr = BufReader::new(stderr);

        loop {
            let (stdout_bytes, stderr_bytes) = match (stdout.fill_buf(), stderr.fill_buf()) {
                (Ok(stdout), Ok(stderr)) => {
                    output.write_all(stdout).expect("Couldn't write");
                    output.write_all(stderr).expect("Couldn't write");

                    (stdout.len(), stderr.len())
                }
                other => panic!("Some better error handling here... {:?}", other)
            };

            if stdout_bytes == 0 && stderr_bytes == 0 {
                // Seems less-than-ideal; should be some way of
                // telling if the child has actually exited vs just
                // not outputting anything.
                break;
            }

            stdout.consume(stdout_bytes);
            stderr.consume(stderr_bytes);
        }
    }

    let status = child.wait().expect("Waiting for child failed");
    println!("Finished with status {:?}", status);
    println!("Combined output: {:?}", std::str::from_utf8(&output))
}

The biggest gap is telling when the process has exited. I'm surprised by the lack of a relevant method on Child.

See also How do I prefix Command stdout with [stdout] and [sterr]?

---

In this solution, there isn't any intrinsic ordering between the file descriptors. As an analogy, imagine two buckets of water. If you empty a bucket and later see that it's been filled up again, you know the second bucket came after the first. However, if you empty two buckets and come back later and both are filled up, you can't tell which bucket was filled first.

The "quality" of interleaving is a matter of how frequently you read from each file descriptor and which file descriptor is read first. If you read a single byte from each in a very tight loop, you might get completely garbled results but these would be the most "accurate" with regard to ordering. Likewise, if a program prints "A" to stderr then "B" to stdout but the shell reads from stdout before stderr, then the result would be "BA", which looks backwards.