Use StringIO as stdin with Popen

问题:

I have the following shell script that I would like to write in Python (of course grep . is actually a much more complex command):

#!/bin/bash

(cat somefile 2>/dev/null || (echo 'somefile not found'; cat logfile)) \
| grep .

I tried this (which lacks an equivalent to cat logfile anyway):

#!/usr/bin/env python

import StringIO
import subprocess

try:
    myfile = open('somefile')
except:
    myfile = StringIO.StringIO('somefile not found')

subprocess.call(['grep', '.'], stdin = myfile)

But I get the error AttributeError: StringIO instance has no attribute 'fileno'.

I know I should use subprocess.communicate() instead of StringIO to send strings to the grep process, but I don't know how to mix both strings and files.

回答1:

p = subprocess.Popen(['grep', '...'], stdin=subprocess.PIPE, 
                                      stdout=subprocess.PIPE)
output, output_err = p.communicate(myfile.read())

回答2:

Don't use bare except, it may catch too much. In Python 3:

#!/usr/bin/env python3
from subprocess import check_output

try:
    file = open('somefile', 'rb', 0)
except FileNotFoundError:
    output = check_output(cmd, input=b'somefile not found')
else:
    with file:
        output = check_output(cmd, stdin=file)

It works for large files (the file is redirected at the file descriptor level -- no need to load it into the memory).

If you have a file-like object (without a real .fileno()); you could write to the pipe directly using .write() method:

#!/usr/bin/env python3
import io
from shutil import copyfileobj
from subprocess import Popen, PIPE
from threading import Thread

try:
    file = open('somefile', 'rb', 0)
except FileNotFoundError:
    file = io.BytesIO(b'somefile not found')

def write_input(source, sink):
    with source, sink:
        copyfileobj(source, sink)

cmd = ['grep', 'o']
with Popen(cmd, stdin=PIPE, stdout=PIPE) as process:
    Thread(target=write_input, args=(file, process.stdin), daemon=True).start()
    output = process.stdout.read()