Shell script while read loop executes only once

Question

I am writing a shell script to batch process .mov files off my camera through Handbrake to save HD space. The script searches a directory with 'find' and then runs Handbrake on each .mov file found, matching the creation date of the resulting file to the source file's date with 'touch'.

I originally did this with a for loop:

for i in $(find "$*" -iname '*.mov') ; do
  ~/Unix/HandbrakeCLI --input "$i" --output "$i".mp4 --preset="Normal"
  touch -r "$i" "$i".mp4
done

This worked, but failed if the input files had spaces in their file names. So I tried a while loop instead:

find "$*" -iname '*.mov' | while read i ; do
  ~/Unix/HandbrakeCLI --input "$i" --output "$i".mp4 --preset="Normal"
  touch -r "$i" "$i".mp4
done

The problem with this loop is that it works for the first file in the directory, and then exits the loop. Note that if I substitute an "echo $i" in the body of the while loop, it prints all of the .mov files in the directory, so the loop is structured correctly.

I believe there is a partial answer to my question on this stackoverflow thread. But the solution is specific to ssh and doesn't solve the general problem. Seems to have something do do with stdin being used by a sub-process, but I don't exactly understand how this works.

Any advice?

I'm on OSX 10.6

Answer 1

You are probably running with a shellopt '-e' (exit on errors)

Try

set +e

Answer 2

I've got the same problem. I think HandbrakeCLI is consuming stdin. I don't have my actual system here, but my testing shows that's what is probably happening.

"output" is a file with the numbers 1-10, each on their own line.

while read line ; do echo "MAIN: $line"; ./consumer.sh; done < output

consumer.sh:

#!/bin/sh

while read line ; do
  echo "CONSUMER: $line"
done

Results:

MAIN: 1
CONSUMER: 2
CONSUMER: 3
CONSUMER: 4
CONSUMER: 5
CONSUMER: 6
CONSUMER: 7
CONSUMER: 8
CONSUMER: 9
CONSUMER: 10

Changing the outer while loop will fix the problem:

while read line ; do echo "MAIN: $line"; ./consumer.sh < /dev/null; done < output

Results:

MAIN: 1
MAIN: 2
MAIN: 3
MAIN: 4
MAIN: 5
MAIN: 6
MAIN: 7
MAIN: 8
MAIN: 9
MAIN: 10

Answer 3

Taken from this answer: I now echo nothing into HandbrakeCLI to ensure it's not using the same stdin as my script:

find . -name "*.mkv" | while read FILE
do
    echo "" | handbrake-touch "$FILE"

    if [ $? != 0 ]
    then
        echo "$FILE had problems" >> errors.log  
    fi
done

...and it works as intended/expected.

Answer 4

This link fixed the problem for me on Ubuntu Linux 11.04 .

Preventing a child process (HandbrakeCLI) from causing the parent script to exit

Here is the code that works for me:

ls -t *.mpeg | tail -n +2 | while read name
do echo "$name" ; in="$name" ; out=coded/"`echo $name | sed 's/.mpeg$/.mkv/'`"
echo "" | HandBrakeCLI -i "$in" -o "$out" -a '1,2' ; mv -v "$name" trash/"$name"
done

Answer 5

Consider just using find to invoke a shell, instead of wrapping find in a while loop.

find "$@" -iname '*.mov' -exec sh -c '
    ~/Unix/HandbrakeCLI --input "$1" --output "$1".mp4 --preset="Normal"
    touch -r "$1" "$1".mp4
' _ {} ';'

Answer 6

One possibility is to use the safe find:

while IFS= read -r -d '' -u 9
do
    ~/Unix/HandbrakeCLI --input "$REPLY" --output "$REPLY".mp4 --preset="Normal"
    touch -r "$REPLY" "$REPLY".mp4
done 9< <( find "$@" -type f -print0 )

This should be POSIX compatible, but only works if neither HandbrakeCLI nor touch read from standard input and no file names contain newlines:

find "$@" -type f -print0 | while IFS= read -r
do
    ~/Unix/HandbrakeCLI --input "$REPLY" --output "$REPLY".mp4 --preset="Normal"
    touch -r "$REPLY" "$REPLY".mp4
done