Read from File, or STDIN

Question

I've written a command line utility that uses getopt for parsing arguments given on the command line. I would also like to have a filename be an optional argument, such as it is in other utilities like grep, cut etc. So, I would like it to have the following usage

tool -d character -f integer [filename]

How can I implement the following?

• if a filename is given, read from the file.
• if a filename is not given, read from STDIN.

Answer 1

To make use of python's with statement, one can use the following code:

import sys
with open(sys.argv[1], 'r') if len(sys.argv) > 1 else sys.stdin as f:
    # read data using f
    # ......

Answer 2

In the simplest terms:

import sys
# parse command line
if file_name_given:
    inf = open(file_name_given)
else:
    inf = sys.stdin

At this point you would use inf to read from the file. Depending on whether a filename was given, this would read from the given file or from stdin.

When you need to close the file, you can do this:

if inf is not sys.stdin:
    inf.close()

However, in most cases it will be harmless to close sys.stdin if you're done with it.

Answer 3

I prefer to use "-" as an indicator that you should read from stdin, it's more explicit:

import sys
with open(sys.argv[1], 'r') if sys.argv[1] is not "-" else sys.stdin as f:
    pass # do something here

Answer 4

The fileinput module may do what you want - assuming the non-option arguments are in args then:

import fileinput
for line in fileinput.input(args):
    print line

If args is empty then fileinput.input() will read from stdin; otherwise it reads from each file in turn, in a similar manner to Perl's while(<>).

Answer 5

Something like:

if input_from_file:
    f = open(file_name, "rt")
else:
    f = sys.stdin
inL = f.readline()
while inL:
    print inL.rstrip()
    inL = f.readline()

Answer 6

I like the general idiom of using a context manager, but the (too) trivial solution ends up closing sys.stdin when you are out of the with statement, which I want to avoid.

Borrowing from this answer, here is a workaround:

import sys
import contextlib

@contextlib.contextmanager
def _smart_open(filename, mode='Ur'):
    if filename == '-':
        if mode is None or mode == '' or 'r' in mode:
            fh = sys.stdin
        else:
            fh = sys.stdout
    else:
        fh = open(filename, mode)
    try:
        yield fh
    finally:
        if filename is not '-':
            fh.close()

if __name__ == '__main__':
    args = sys.argv[1:]
    if args == []:
        args = ['-']
    for filearg in args:
        with _smart_open(filearg) as handle:
            do_stuff(handle)

I suppose you could achieve something similar with os.dup() but the code I cooked up to do that turned out to be more complex and more magical, whereas the above is somewhat clunky but very straightforward.