'yes' reporting error with subprocess comm

Question

I am using the following function to run a command in Python:

def run_proc(cmd):
    child = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    stdout, stderr = child.communicate()
    returncode = child.returncode
    return stdout, stderr, returncode

It has always been working fine, however now I'm trying to use the yes program to pipe output to stdin. The command I'm trying to run is the following:

yes '' | apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" dist-upgrade

but I believe it could be substituted with a general example, like:

yes | head -3 | cat

My problem is that if I try to run any command which has yes | in it, the above subprocess.Popen will contain the error messages:

yes: standard output: Broken pipe
yes: write error

For me it seems that the piping still works, as can be seen from yes | head -3 | cat's answer: y y y.

I have the following questions:

1. Is the yes piping still functional, even though yes reports error?
2. How can I fix it?

Answer 1

Saying:

yes | head -3

causes head to send a SIGPIPE to yes once it's done reading 3 lines of input, i.e. it'd send a signal to terminate yes.

$ yes | head -3
y
y
y
$ echo "${PIPESTATUS[@]}"
141 0

The solution would be to avoid a SIGPIPE!

Answer 2

the other question answers the why ... I'll try and give you a work around

could you not do something like

proc = subprocess.Popen(cmd, shell=True, 
                             stdout=subprocess.PIPE, 
                             stderr=subprocess.PIPE,
                             stdin = subprocess.PIPE)

for i in range(10):  #send 10 y's
    time.sleep(1) # 1 second apart
    proc.stdin.write("y") #you may also need to send a newline ...

print proc.communicate()

see below (I didnt bother with the delay since head isnt really doing much)

>>> import subprocess
>>> proc = subprocess.Popen("head -3",
...                          shell = True,
...                          stdout = subprocess.PIPE,
...                          stderr=subprocess.PIPE,
...                          stdin=subprocess.PIPE)
>>> for i in range(10):
...    proc.stdin.write("y\n")
...
>>> proc.communicate()
('y\ny\ny\n', '')

Answer 3

The issue is that subprocess module before Python 3.2+ doesn't restore SIGPIPE signal handler to default action. That is why you get EPIPE write error instead.

In Python 3.2+

>>> from subprocess import check_output
>>> check_output("yes | head -3", shell=True)
b'y\ny\ny\n'

yes is killed by SIGPIPE when head exits.

In Python 2:

>>> from subprocess import check_output
>>> check_output("yes | head -3", shell=True)
yes: standard output: Broken pipe
yes: write error
'y\ny\ny\n'

yes got EPIPE write error. It is safe to ignore the error. It communicates the same information as SIGPIPE.

To workaround the problem, you could emulate restore_signals in Python 2 using preexec_fn parameter :

>>> from subprocess import check_output
>>> import signal
>>> def restore_signals(): # from http://hg.python.org/cpython/rev/768722b2ae0a/
...     signals = ('SIGPIPE', 'SIGXFZ', 'SIGXFSZ')
...     for sig in signals:
...         if hasattr(signal, sig):
...            signal.signal(getattr(signal, sig), signal.SIG_DFL)
... 
>>> check_output("yes | head -3", shell=True, preexec_fn=restore_signals)
'y\ny\ny\n'