Synchronize shell script execution

Question

A modified version of a shell script converts an audio file from FLAC to MP3 format. The computer has a quad-core CPU. The script is run using:

./flac2mp3.sh $(find flac -type f)

This converts the FLAC files in the flac directory (no spaces in file names) to MP3 files in the mp3 directory (at the same level as flac). If the destination MP3 file already exists, the script skips the file.

The problem is that sometimes two instances of the script check for the existence of the same MP3 file at nearly the same time, resulting in mangled MP3 files.

How would you run the script multiple times (i.e., once per core), without having to specify a different file set on each command-line, and without overwriting work?

Update - Minimal Race Condition

The script uses the following locking mechanism:

  # Convert FLAC to MP3 using tags from flac file.
  #
  if [ ! -e $FLAC.lock ]; then
    touch $FLAC.lock
    flac -dc "$FLAC" | lame${lame_opts} \
      --tt "$TITLE" \
      --tn "$TRACKNUMBER" \
      --tg "$GENRE" \
      --ty "$DATE" \
      --ta "$ARTIST" \
      --tl "$ALBUM" \
      --add-id3v2 \
      - "$MP3"
    rm $FLAC.lock
  fi;

However, this still leaves a race condition.

Answer 1

Use a tool like FLOM (Free LOck Manager) and simply serialize your command as below:

flom -- flac ....

Answer 2

To lock a file process you can create a file with the same name with a .lock extension.

Before starting the encoding check the existence of the .lock file, and optionally make sure the date of the lockfile isn't too old (in case the process dies). If it does not exist, create it before the encoding starts, and remove it after the encoding is complete.

You can also flock the file, but this only really works in c where you are calling flock() and writing to the file then closing and unlocking. For a shell script, you probably are calling another utility to do the writing of the file.