How to close std-streams from java.lang.Process ap

问题:

This question is about java.lang.Process and its handling of stdin, stdout and stderr.

We have a class in our project that is an extension to org.apache.commons.io.IOUtils. There we have a quiet new method for closing the std-streams of a Process-Object appropriate? Or is it not appropriate?

/**
 * Method closes all underlying streams from the given Process object.
 * If Exit-Code is not equal to 0 then Process will be destroyed after 
 * closing the streams.
 *
 * It is guaranteed that everything possible is done to release resources
 * even when Throwables are thrown in between.
 *
 * In case of occurances of multiple Throwables then the first occured
 * Throwable will be thrown as Error, RuntimeException or (masked) IOException.
 *
 * The method is null-safe.
 */
public static void close(@Nullable Process process) throws IOException {
    if(process == null) {
      return;
    }

    Throwable t = null;

    try {
      close(process.getOutputStream());
    }
    catch(Throwable e) {
      t = e;
    }

    try{
      close(process.getInputStream());
    }
    catch(Throwable e) {
      t = (t == null) ? e : t;
    }

    try{
      close(process.getErrorStream());
    }
    catch (Throwable e) {
      t = (t == null) ? e : t;
    }

    try{
      try {
        if(process.waitFor() != 0){
          process.destroy();
        }
      }
      catch(InterruptedException e) {
        t = (t == null) ? e : t;
        process.destroy();
      }
    }
    catch (Throwable e) {
      t = (t == null) ? e : t;
    }

    if(t != null) {
      if(t instanceof Error) {
        throw (Error) t;
      }

      if(t instanceof RuntimeException) {
        throw (RuntimeException) t;
      }

      throw t instanceof IOException ? (IOException) t : new IOException(t);
    }
}

---

public static void closeQuietly(@Nullable Logger log, @Nullable Process process) {
  try {
    close(process);
  }
  catch (Exception e) {
    //log if Logger provided, otherwise discard
    logError(log, "Fehler beim Schließen des Process-Objekts (inkl. underlying streams)!", e);
  }
}

---

public static void close(@Nullable Closeable closeable) throws IOException {
  if(closeable != null) {
    closeable.close();
  }
}

Methods like these are basically used in finally-blocks.

What I really want to know is if I am safe with this implementation? Considering things like: Does a process object always return the same stdin, stdout and stderr streams during its lifetime? Or may I miss closing streams previously returned by process' getInputStream(), getOutputStream() and getErrorStream() methods?

There is a related question on StackOverflow.com: java: closing subprocess std streams?

Edit

As pointed out by me and others here:

• InputStreams have to be totally consumed. When not done then the subprocess may not terminate, because there is outstanding data in its output streams.
• All three std-streams have to be closed. Regardless if used before or not.
• When the subprocess terminates normally everything should be fine. When not then it have to be terminated forcibly.
• When an exit code is returned by subprocess then we do not need to destroy() it. It has terminated. (Even when not necessarily terminated normally with Exit Code 0, but it terminated.)
• We need to monitor waitFor() and interrupt when timeout exceeds to give process a chance to terminate normally but killing it when it hangs.

Unanswered parts:

• Consider Pros and Cons of consuming the InputStreams in parallel. Or must they be consumed in particular order?

回答1:

An attempt at simplifying your code:

public static void close(@Nullable Process process) throws IOException
{
    if(process == null) { return; }

    try
    {
        close(process.getOutputStream());
        close(process.getInputStream());
        close(process.getErrorStream());

        if(process.waitFor() != 0)
        {
            process.destroy();
        }
    }
    catch(InterruptedException e)
    {
        process.destroy();
    }
    catch (RuntimeException e)
    {
        throw (e instanceof IOException) ? e : new IOException(e);
    }
}

By catching Throwable I assume you wish to catch all unchecked exceptions. That is either a derivative of RuntimeException or Error. However Error should never be catched, so I have replaced Throwable with RuntimeException.

(It is still not a good idea to catch all RuntimeExceptions.)

回答2:

As the question you linked to states, it is better to read and discard the output and error streams. If you are using apache commons io, something like,

new Thread(new Runnable() {public void run() {IOUtils.copy(process.getInputStream(), new NullOutputStream());}}).start();
new Thread(new Runnable() {public void run() {IOUtils.copy(process.getErrorStream(), new NullOutputStream());}}).start();

You want to read and discard stdout and stderr in a separate thread to avoid problems such as the process blocking when it writes enough info to stderr or stdout to fill the buffer.

If you are worried about having two many threads, see this question

I don't think you need to worry about catching IOExceptions when copying stdout, stdin to NullOutputStream, since if there is an IOException reading from the process stdout/stdin, it is probably due to the process being dead itself, and writing to NullOutputStream will never throw an exception.

You don't need to check the return status of waitFor().

Do you want to wait for the process to complete? If so, you can do,

while(true) {
     try
     {
         process.waitFor();
         break;
     } catch(InterruptedException e) {
         //ignore, spurious interrupted exceptions can occur
     }

}

Looking at the link you provided you do need to close the streams when the process is complete, but destroy will do that for you.

So in the end, the method becomes,

public void close(Process process) {

    if(process == null) return;

    new Thread(new Runnable() {public void run() {IOUtils.copy(process.getInputStream(), new NullOutputStream());}}).start();
    new Thread(new Runnable() {public void run() {IOUtils.copy(process.getErrorStream(), new NullOutputStream());}}).start();
    while(true) {
        try
        {
            process.waitFor();
            //this will close stdin, stdout and stderr for the process
            process.destroy();
            break;
        } catch(InterruptedException e) {
            //ignore, spurious interrupted exceptions can occur
        }

   }
}

回答3:

Just to let you know what I have currently in our codebase:

public static void close(@Nullable Process process) throws IOException {
  if (process == null) {
    return;
  }

  Throwable t = null;

  try {
    flushQuietly(process.getOutputStream());
  }
  catch (Throwable e) {
    t = mostImportantThrowable(t, e);
  }

  try {
    close(process.getOutputStream());
  }
  catch (Throwable e) {
    t = mostImportantThrowable(t, e);
  }

  try {
    skipAllQuietly(null, TIMEOUT, process.getInputStream());
  }
  catch (Throwable e) {
    t = mostImportantThrowable(t, e);
  }

  try {
    close(process.getInputStream());
  }
  catch (Throwable e) {
    t = mostImportantThrowable(t, e);
  }

  try {
    skipAllQuietly(null, TIMEOUT, process.getErrorStream());
  }
  catch (Throwable e) {
    t = mostImportantThrowable(t, e);
  }

  try {
    close(process.getErrorStream());
  }
  catch (Throwable e) {
    t = mostImportantThrowable(t, e);
  }

  try {
    try {
      Thread monitor = ThreadMonitor.start(TIMEOUT);
      process.waitFor();
      ThreadMonitor.stop(monitor);
    }
    catch (InterruptedException e) {
      t = mostImportantThrowable(t, e);
      process.destroy();
    }
  }
  catch (Throwable e) {
    t = mostImportantThrowable(t, e);
  }

  if (t != null) {
    if (t instanceof Error) {
      throw (Error) t;
    }

    if (t instanceof RuntimeException) {
      throw (RuntimeException) t;
    }

    throw t instanceof IOException ? (IOException) t : new IOException(t);
  }
}

---

skipAllQuietly(...) consumes complete InputStreams. It uses internally an implementation similar to org.apache.commons.io.ThreadMonitor to interrupt consumption if a given timeout exceeded.

mostImportantThrowable(...) decides over what Throwable should be returned. Errors over everything. First occured higher prio than later occured. Nothing very important here since these Throwable are most probably discarded anyway later. We want to go on working here and we can only throw one, so we have to decide what we throw at the end, if ever.

close(...) are null-safe implementations to close stuff but throwing Exception when something went wrong.