bash: How do I ensure termination of process substitution used with exec?

bash, process-substitution

Solution

The following works in bash 4, using coprocesses:

#!/bin/bash
fd_re='^[0-9]+$'
cleanup_and_wait() {
    if [[ ${COPROC[1]} =~ $fd_re ]] ; then
        eval "exec ${COPROC[1]}<&-"
        echo "waiting for $filename to finish" >&2
        wait $COPROC_PID
    fi
}

while IFS= read -r line; do
    case $line in
    @*)
        cleanup_and_wait
        filename=${line:1}
        echo "starting $filename" >&2
        coproc { sort >"$filename"; echo "Finished with $filename" >&2; }
        ;;
    *)
        printf '%s\n' "$line" >&${COPROC[1]}
        ;;
    esac
done
cleanup_and_wait

For prior versions of bash, a named pipe can be used instead:

cleanup_and_wait() {
    if [[ $child_pid ]] ; then
      exec 4<&-
      echo "waiting for $filename to finish" >&2
      wait $child_pid
    fi
}

# this is a bit racy; without a force option to mkfifo,
# however, the race is unavoidable
fifo_name=$(mktemp -u -t fifo.XXXXXX)
if ! mkfifo "$fifo_name" ; then
  echo "Someone else may have created our temporary FIFO before we did!" >&2
  echo "This can indicate an attempt to exploit a race condition as a" >&2
  echo "security vulnarability and should always be tested for." >&2
  exit 1
fi

# ensure that we clean up even on unexpected exits
trap 'rm -f "$fifo_name"' EXIT

while IFS= read -r line; do
    case $line in
    @*)
        cleanup_and_wait
        filename=${line:1}
        echo "starting $filename" >&2
        { sort >"$filename"; echo "finished with $filename" >&2; } <"$fifo_name" &
        child_pid=$!
        exec 4>"$fifo_name"
        ;;
    *)
        printf '%s\n' "$line" >&4
        ;;
    esac
done
cleanup_and_wait

A few notes:

- It's safer to use `printf '%s\n' "$line"` than `echo "$line"`; if a line contains only `-e`, for instance, some versions of `echo` will do nothing with it.

- Using an EXIT trap for cleanup ensures that an unexpected SIGTERM or other error won't leave the stale fifo sitting around.

- If your platform provides a way to create a FIFO with an unknown name in a single, atomic operation, use it; this would avoid the condition that requires us to always test whether the mkfifo is successful.

Problem

If I run ``` $#/bin/bash for i in `seq 5`; do exec 3> >(sed -e "s/^/$i: /"; echo "$i-") echo foo >&3 echo bar >&3 exec 3>&- done ``` then the result is not synchronous; it could be something like: ``` 1: foo 1: bar 2: foo 2: bar 1- 3: foo 3: bar 2- 3- 4: foo 5: foo 4: bar 5: bar 4- 5- ``` How do I ensure that the process substitution `>(...)` is completed before proceeding to the next iteration? Inserting `sleep 0.1` after `exec 3>&-` helped, but it's inelegant, inefficient, and not guaranteed to always work. EDIT: The example may look silly, but it was for illustration only. What I'm doing is reading a stream of input in a loop, feeding each line to a process which occasionally changes during the loop. Easier explained in code: ``` # again, simplified for illustration while IFS= read line; do case $line in @*) exec 3>&- filename=${line:1} echo "starting $filename" exec 3> >(sort >"$filename"; echo "finished $filename") ;; *) echo "$line" >&3 ;; esac done exec 3>&- ```

Original source