C# System.Diagnostics.Process redirecting Standard

Question

I running an exe from a .NET app and trying to redirect standard out to a streamreader. The problem is that when I do

myprocess.exe >> out.txt

out.txt is close to 14mb. When I do the command line version it is very fast but when I run the process from my csharp app it is excruciatingly slow because I believe the default streamreader flushes every 4096 bytes.

Is there a way to change the default stream reader for the Process object?

Answer 1

The Process class exposes the stdout stream directly, so you should be able to read it at whatever pace you like. It's probably best to read it in small chunks and avoid calling ReadToEnd.

For example:

using(StreamReader sr = new StreamReader(myProcess.StandardOutput))
{
  string line;
  while((line = sr.ReadLine()) != null)
  {
    // do something with line
  }
}

Answer 2

I haven't tried, but it looks like the asynchronous methods may offer better performance. Instead of using process.StandardOutput, try this method instead:

Process process = Process
    .Start(new ProcessStartInfo("a.exe"){RedirectStandardOutput = true});
if (process != null)
{
    process.OutputDataReceived += ((sender, e) =>
                                   {
                                       string consoleLine = e.Data;
                                       //handle data
                                   });
    process.BeginOutputReadLine();
}

Answer 3

This worked out for me:

            var sb = new StringBuilder();
            while (!proc.StandardOutput.EndOfStream)
            {
                sb.Append(proc.StandardOutput.ReadToEnd());
                proc.StandardOutput.DiscardBufferedData();
            }

Answer 4

Edit: Just realized I'm answering the wrong question. In my case the stdout buffer was full and WaitForExit() was blocking forever, because nothing was reading from the buffer yet. So if you have THAT problem, then here's a solution. ;)

This is my first day with C# so please understand that this might not be the best solution, and might not always work. But it works in the 2x I've tested it. ;) This is synchronous, just start start writing the redirected stdout/stderr to the file before you WaitForExit(). This way WaitForExit() won't block waiting for the stdout buffer to be emptied.

      string str_MyProg = "my.exe";
      string str_CommandArgs = " arg1 arg2"'
      System.Diagnostics.ProcessStartInfo procStartInfo = new System.Diagnostics.ProcessStartInfo(str_MyProg, str_CommandArgs);

      procStartInfo.RedirectStandardError = true;
      procStartInfo.RedirectStandardOutput = true; // Set true to redirect the process stdout to the Process.StandardOutput StreamReader
      procStartInfo.UseShellExecute = false;
      procStartInfo.CreateNoWindow = true;          // Do not create the black window

      // Create a process, assign its ProcessStartInfo and start it
      System.Diagnostics.Process myProcess = new System.Diagnostics.Process();
      myProcess.StartInfo = procStartInfo;
      myProcess.Start();

      // Dump the output to the log file
      string stdOut = myProcess.StandardOutput.ReadToEnd();
      StreamWriter logFile = new StreamWriter("output.txt" );
      logFile.Write(stdOut);
      logFile.Close();

      myProcess.WaitForExit();          

Answer 5

Yes, that's about right. There is a buffer that stores the process output, usually between 1 and 4KB in the common CRT implementations. One small detail: that buffer is located in the process you start, not the .NET program.

Nothing very special needs to happen when you redirect to a file, the CRT directly writes it. But if you redirect to your .NET program then output goes from the buffer into a pipe. Which then takes a thread switch to your program so you can empty the pipe. Back and forth a good 700 times.

Yes, not fast. Easily fixed though, call setvbuf() in the program you are running to increase the stdout and stderr output buffer sizes. Then again, that takes having the source code of that program.

Anticipating a problem with that: maybe you ought to use cmd.exe /c to get the redirection to a file, then read the file.