Synchronized reading data from Process's empty

Question

This question already has an answer here:

• Reading from a process, StreamReader.Peek() not working as expected 1 answer

I am having trouble setting up a c# application that creates and interacts with a python process1. A simplistic example is given below.

EDIT: Further research on SO unveiled that my question is a possible duplicate. A potentially related known bug in the .NET Framework is duscussed here and here. It seems that back in 2014 the only easy workaround is indeed to ask the child process to write something in both stdOut and stdErr. But I'd like to know if this assumption is correct and wonder if there hasn't been a fix since 2014?

I have to fulfill the following boundary conditions:

1. I am not able to close the python process after handing over a script or a command, but I have to keep the process alive. Edit: For that reason I can not make use of the Process.WaitForExit() Method
2. As the std's remain open all the time, I believe I can't check for EndOfStream, as that would require to read to the end of the stream, which does not exist.
3. Furthermore, my application has to wait for the response of the python process, therefore the asynchronous option using BeginOutputReadLine() with OnOutputDataReceived seems not appropriate to me.
4. As the commands that will be sent to python are arbitrary user input, pythons result might be either in stdOut or stdErr ("4+7" results in "11" stored in stdOut; "4+a" results in "name 'a' is not defined" in stdErr)

What I do is to:

• set up a python process in interactive mode (Argument "-i")
• enable redirect of StdIn, Out, and Err
• start the process
• get StreamReaders and Writers for the Std's

After that, I want to initially check the StdOut and StdErr. I know that python writes the following piece of information to the StdErr

Python 2.7.11 (v2.7.11:6d1b6a68f775, Dec 5 2015, 20:32:19) [MSC v.1500 32 bit (Intel)] on win32

and I am able to get this line by using errorReader.Peek() and reading character-based from the errorReader 2.

However, the situation with another process might be totally different. Even with Python, I run into the following problem: when I want to initially read from the outputReader, there is nothing contained in it and outputReader.Peek() seems to run into a deadlock. As mentioned above, the same holds for outputReader.EndOfStream or outputReader.ReadToEnd(). So how do I know if the stdOut can be used at all without causing a deadlock?

Code:

        // create the python process StartupInfo object
        ProcessStartInfo _tempProcessStartInfo = new ProcessStartInfo(@"C:\TMP\Python27\python.exe");
        // ProcessStartInfo _tempProcessStartInfo = new ProcessStartInfo(PathToPython + "python.exe");

        // python uses "-i" to run in interactive mode
        _tempProcessStartInfo.Arguments = "-i";

        // Only start the python process, but don't show a (console) window
        _tempProcessStartInfo.WindowStyle = ProcessWindowStyle.Minimized;
        _tempProcessStartInfo.CreateNoWindow = true;


        // Enable the redirection of python process std's
        _tempProcessStartInfo.UseShellExecute = false;

        _tempProcessStartInfo.RedirectStandardOutput = true;
        _tempProcessStartInfo.RedirectStandardInput = true;
        _tempProcessStartInfo.RedirectStandardError = true;

        // Create the python process object and apply the startupInfos from above
        Process _tempProcess = new Process();
        _tempProcess.StartInfo = _tempProcessStartInfo;

        // Start the process
        bool _hasStarted = _tempProcess.Start();

        //// ASynch reading seems not appropriate to me:
        // _tempProcess.BeginOutputReadLine();
        // _tempProcess.BeginErrorReadLine();

        // Create StreamReaders and Writers for the Std's
        StreamReader outputReader = _tempProcess.StandardOutput;
        StreamReader errorReader = _tempProcess.StandardError;
        StreamWriter commandWriter = _tempProcess.StandardInput;

        // Create StringBuilder that collects results and ErrorMessages
        StringBuilder tmp = new StringBuilder("");

        // Create temp variable that is used to peek into streams. C# uses -1 to indicate that there is no more byte to read
        int currentPeek = -1;

        // Get Initial Error Message. In this specific case, this is the python version
        tmp.AppendLine("INITIAL ERROR MESSAGE:");
        currentPeek = errorReader.Peek();
        while (currentPeek >= 0)
        {
            char text = (char)errorReader.Read();
            tmp.Append(text);
            currentPeek = errorReader.Peek();
        }

        // Get initial output Message. In this specific case, this is EMPTY, which seems to cause this problem, as ...
        tmp.AppendLine("INITIAL STDOUT MESSAGE:");

        //// ... the following command CREATES a well defined output, and afterwards everything works fine (?) but ...
        //commandWriter.WriteLine(@"print 'Hello World'");

        //// ... without the the above command, neither
        //bool isEndOfStream = outputReader.EndOfStream;
        //// ... nor
        // currentPeek = outputReader.Peek();
        //// ... nor
        // tmp.AppendLine(outputReader.ReadLine());
        //// ... nor 
        //tmp.AppendLine(outputReader.ReadToEnd());
        //// ... works

        // Therefore, the following command creates a deadlock
        currentPeek = outputReader.Peek();
        while (currentPeek >= 0)
        {

            char text = (char)outputReader.Read();
            tmp.Append(text);
            currentPeek = errorReader.Peek();
        }

        _currentPythonProcess = _tempProcess;
        return true;

1 An easy fix to this very specific problem is to send a valid command to the process first, for example simply "4", which returns a "4" as well... However, I want to understand how process streams, pipes and the corresponing readers and writers work and how I can use them in C#. Who knows what future brings, maybe I run into buffer problems when pythons response is 2^n+1 bytes long... 2 I know that I can also read line-based. However, the Peek() prevents me from reporting problems that are related to truncated lines.

Answer 1

If you can wait for the process to end and then read the buffers, you might be able to use Process.WaitForExit. There is also another method you can check, Process.WaitForInputIdle, but it depends on the process having a message loop, which I don't think a Python script gets when executing.