Synchronize shell script execution
bash, linux, multiprocessing, race-condition, shell
Solution
The "lockfile" command provides what you're trying to do for shell scripts without the race condition. The command was written by the procmail folks specifically for this sort of purpose and is available on most BSD/Linux systems (as procmail is available for most environments).
Your test becomes something like this:
lockfile -r 3 $FLAC.lock
if test $? -eq 0 ; then
flac -dc "$FLAC" | lame${lame_opts} \
--tt "$TITLE" \
--tn "$TRACKNUMBER" \
--tg "$GENRE" \
--ty "$DATE" \
--ta "$ARTIST" \
--tl "$ALBUM" \
--add-id3v2 \
- "$MP3"
fi
rm -f $FLAC.lock
Alternatively, you could make lockfile keep retrying indefinitely so you don't need to test the return code, and instead can test for the output file for determining whether to run flac.
Problem
A modified version of a shell script converts an audio file from FLAC to MP3 format. The computer has a quad-core CPU. The script is run using: ``` ./flac2mp3.sh $(find flac -type f) ``` This converts the FLAC files in the `flac` directory (no spaces in file names) to MP3 files in the `mp3` directory (at the same level as `flac`). If the destination MP3 file already exists, the script skips the file. The problem is that sometimes two instances of the script check for the existence of the same MP3 file at nearly the same time, resulting in mangled MP3 files. How would you run the script multiple times (i.e., once per core), without having to specify a different file set on each command-line, and without overwriting work? Update - Minimal Race Condition The script uses the following locking mechanism: ``` # Convert FLAC to MP3 using tags from flac file. # if [ ! -e $FLAC.lock ]; then touch $FLAC.lock flac -dc "$FLAC" | lame${lame_opts} \ --tt "$TITLE" \ --tn "$TRACKNUMBER" \ --tg "$GENRE" \ --ty "$DATE" \ --ta "$ARTIST" \ --tl "$ALBUM" \ --add-id3v2 \ - "$MP3" rm $FLAC.lock fi; ``` However, this still leaves a race condition.