Iterate through folders, then subfolders and print

Question

I am trying to use python to create the files needed to run some other software in batch. For part of this I need to produce a text file that loads the needed data files into the software. My problem is that the files I need to enter into this text file are stored in a set of structured folders.

I need to loop over a set of folders (up to 20), which each could contain up to 3 more folders which contain the files I need. The bottom level of the folders contain a set of files needed for each run of the software. The text file should have the path+name of these files printed line by line, add an instruction line and then move to the next set of files from a folder and so on until all of sub level folders have been checked.

Answer 1

Use os.walk(). The following will output a list of all files within the subdirectories of "dir". The results can be manipulated to suit you needs:

import os                                                                                                             

def list_files(dir):                                                                                                  
    r = []                                                                                                            
    subdirs = [x[0] for x in os.walk(dir)]                                                                            
    for subdir in subdirs:                                                                                            
        files = os.walk(subdir).next()[2]                                                                             
        if (len(files) > 0):                                                                                          
            for file in files:                                                                                        
                r.append(subdir + "/" + file)                                                                         
    return r                                                                                                          

Answer 2

Charles' answer is good, but can be improved upon to increase speed and efficiency. Each item produced by os.walk() is a tuple of three items. Those items are:

1. The working directory
2. A list of strings naming any sub-directories present in the working directory
3. A list of files present in the working directory

Knowing this, much of Charles' code can be condensed with the modification of a forloop:

import os

def list_files(dir):
    r = []
    for root, dirs, files in os.walk(dir):
        for name in files:
            r.append(os.path.join(root, name))
    return r