How to loop through file names returned by find?

Question

x=$(find . -name "*.txt")
echo $x

if I run the above piece of code in Bash shell, what I get is a string containing several file names separated by blank, not a list.

Of course, I can further separate them by blank to get a list, but I'm sure there is a better way to do it.

So what is the best way to loop through the results of a find command?

Answer 1

Filenames can include spaces and even control characters. Spaces are (default) delimiters for shell expansion in bash and as a result of that x=$(find . -name "*.txt") from the question is not recommended at all. If find gets a filename with spaces e.g. "the file.txt" you will get 2 separated strings for processing, if you process x in a loop. You can improve this by changing delimiter (bash IFS Variable) e.g. to \r\n, but filenames can include control characters - so this is not a (completely) safe method.

From my point of view, there are 2 recommended (and safe) patterns for processing files:

1. Use for loop & filename expansion:

for file in ./*.txt; do
    [[ ! -e $file ]] && continue  # continue, if file does not exist
    # single filename is in $file
    echo "$file"
    # your code here
done

2. Use find-read-while & process substitution

while IFS= read -r -d '' file; do
    # single filename is in $file
    echo "$file"
    # your code here
done < <(find . -name "*.txt" -print0)

Remarks

on Pattern 1:

1. bash returns the search pattern ("*.txt") if no matching file is found - so the extra line "continue, if file does not exist" is needed. see Bash Manual, Filename Expansion
2. shell option nullglob can be used to avoid this extra line.
3. "If the failglob shell option is set, and no matches are found, an error message is printed and the command is not executed." (from Bash Manual above)
4. shell option globstar: "If set, the pattern ‘**’ used in a filename expansion context will match all files and zero or more directories and subdirectories. If the pattern is followed by a ‘/’, only directories and subdirectories match." see Bash Manual, Shopt Builtin
5. other options for filename expansion: extglob, nocaseglob, dotglob & shell variable GLOBIGNORE

on Pattern 2:

1. filenames can contain blanks, tabs, spaces, newlines, ... to process filenames in a safe way, find with -print0 is used: filename is printed with all control characters & terminated with NUL. see also Gnu Findutils Manpage, Unsafe File Name Handling, safe File Name Handling, unusual characters in filenames. See David A. Wheeler below for detailed discussion of this topic.

2. There are some possible patterns to process find results in a while loop. Others (kevin, David W.) have shown how to do this using pipes:

   files_found=1 find . -name "*.txt" -print0 | while IFS= read -r -d '' file; do # single filename in $file echo "$file" files_found=0 # not working example # your code here done [[ $files_found -eq 0 ]] && echo "files found" || echo "no files found"

   When you try this piece of code, you will see, that it does not work: files_found is always "true" & the code will always echo "no files found". Reason is: each command of a pipeline is executed in a separate subshell, so the changed variable inside the loop (separate subshell) does not change the variable in the main shell script. This is why I recommend using process substitution as the "better", more useful, more general pattern.
   See I set variables in a loop that's in a pipeline. Why do they disappear... (from Greg's Bash FAQ) for a detailed discussion on this topic.

Additional References & Sources:

• Gnu Bash Manual, Pattern Matching

• Filenames and Pathnames in Shell: How to do it Correctly, David A. Wheeler

• Why you don't read lines with "for", Greg's Wiki

• Why you shouldn't parse the output of ls(1), Greg's Wiki

• Gnu Bash Manual, Process Substitution

Answer 2

You can store your find output in array if you wish to use the output later as:

array=($(find . -name "*.txt"))

Now to print the each element in new line, you can either use for loop iterating to all the elements of array, or you can use printf statement.

for i in ${array[@]};do echo $i; done

or

printf '%s\n' "${array[@]}"

---

You can also use:

for file in "`find . -name "*.txt"`"; do echo "$file"; done

This will print each filename in newline

To only print the find output in list form, you can use either of the following:

find . -name "*.txt" -print 2>/dev/null

or

find . -name "*.txt" -print | grep -v 'Permission denied'

This will remove error messages and only give the filename as output in new line.

If you wish to do something with the filenames, storing it in array is good, else there is no need to consume that space and you can directly print the output from find.

Answer 3

I like to use find which is first assigned to variable and IFS switched to new line as follow:

FilesFound=$(find . -name "*.txt")

IFSbkp="$IFS"
IFS=$'\n'
counter=1;
for file in $FilesFound; do
    echo "${counter}: ${file}"
    let counter++;
done
IFS="$IFSbkp"

Just in case you would like to repeat more actions on the same set of DATA and find is very slow on your server (I/0 high utilization)

Answer 4

With any $SHELL that supports it (sh/bash/zsh/...):

find . -name "*.txt" -exec $SHELL -c '
    echo "$0"
' {} \;

Done.

Answer 5

Assuming you don't have filenames with embedded newlines, you can get a list like this:

list=($(find . -name '*.txt'))
printf '%s\n' "${list[@]}"

As other people have pointed out, whether this is useful depends on the context.

Answer 6

based on other answers and comment of @phk, using fd #3:
(which still allows to use stdin inside the loop)

while IFS= read -r f <&3; do
    echo "$f"

done 3< <(find . -iname "*filename*")