How to “with open” a list of files and obtain thei

Question

This question already has an answer here:

• How can I open multiple files (number of files unknown beforehand) using “with open” statement? 3 answers

Is it possible to with open() all files contained in a list and create file handles for writing?

For example, if my function accepts a list of filenames for data-splitting in a machine learning task,

fname_list = ['train_dataset.txt', 'validate_dataset.txt', 'test_dataset.txt']

then it would be convenient to be able to do:

with open('source_dataset.txt) as src_file, open(name_list, 'w') as <DONT_KNOW_WHAT_TO_DO_HERE>:

And perform some data splitting within the block.

Edit: So my question is basically "Is it possible to obtain multiple file handles for a list of files opened with 'with open()'?"

Answer 1

In Python 3.3 and higher, contextlib.ExitStack can be used to do this correctly and nicely:

from contextlib import ExitStack

with open('source_dataset.txt') as src_file, ExitStack() as stack:
    files = [stack.enter_context(open(fname, 'w')) for fname in fname_list]
    ... do stuff with src_file and the values in files ...
... src_file and all elements in stack cleaned up on block exit ...

Answer 2

You can define a class openfiles to support the with statement:

class openfiles:
    def __init__(self, filelist, mode='r'):
        self.fhandles = [open(f, mode) for f in filelist]

    def __enter__(self):
        return self.fhandles

    def __exit__(self, type, value, traceback):
        map(file.close, self.fhandles)

Then you can:

with openfiles(['file1', 'file2']) as files:
    for f in files:
        print(f.read())