subprocess stdin buffer not flushing on newline wi

Question

I have two small python files, the first reads a line using input and then prints another line

a = input()
print('complete')

The second attempts to run this as a subprocess

import subprocess

proc = subprocess.Popen('./simp.py',
                        stdout=subprocess.PIPE,
                        stdin=subprocess.PIPE,
                        bufsize=1)
print('writing')
proc.stdin.write(b'hey\n')
print('reading')
proc.stdout.readline()

The above script will print "writing" then "reading" but then hang. At first I thought this was a stdout buffering issue, so I changed bufsize=1 to bufsize=0, and this does fix the problem. However, it seems it's the stdin that's causing the problem.

With bufsize=1, if I add proc.stdin.flush() below the write, the process continues. Both of these approaches seem clumsy since (1) unbuffered streams are slow (2) adding flushes everywhere is error-prone. Why does the above write not flush on a newline? The docs say that bufsize is used when creating stdin, stdout, and stderr stream for the subprocess, so what's causing the write to not flush on the newline?

Answer 1

From the docs: "1 means line buffered (only usable if universal_newlines=True i.e., in a text mode)". This works:

import subprocess

proc = subprocess.Popen('./simp.py',
                        stdout=subprocess.PIPE,
                        stdin=subprocess.PIPE,
                        bufsize=1,
                        universal_newlines=True)

print('writing')
proc.stdin.write('hey\n')
print('reading')
proc.stdout.readline()