Stdout race condition between script and subscript

Question

I'm trying to call a script deepScript and process its output within another script shallowScript ; it looks schematically like the following pieces of code:

shallowScript.sh

#!/bin/zsh
exec 1> >( tr "[a-z]" "[A-Z]" )
print "Hello - this is shallowScript"
. ./deepScript.sh

deepScript.sh

#!/bin/zsh
print "Hello - this is deepScript"

Now, when I run ./shallowScript.sh, the outcome is erratic : either it works as expected (very rarely), or it prints an empty line followed by the two expected lines (sometimes), or it prints the two lines and then hangs until I hit return and give it a newline (most of the time). So far, I found out the following:

• it is probably a race condition, as the two "print"s try to output to stdout at the same time; inserting "sleep 1" before the call to ". ./deepScript.sh" corrects the problem consistently
• the problem comes from the process substitution "exec 1> >(tr ...)"; commenting it out also corrects the problem consistently

I've browsed so many forums and posts about process substitution and redirection, but could not find out how to guarantee that my script calls commands synchronously. Ideas ?

zsh --version                                                                                                                                                                   
zsh 5.0.5 (x86_64-apple-darwin14.0)

[EDIT]

As it seems that this strategy is bound to fail or lead to horrible workaround syntax, here is another strategy that seems to work with a bearable syntax: I removed all the redirect from shallowScript.sh and created a third script where the output processing happens in a function:

shallowScript.sh

#!/bin/zsh
print "Hello - this is shallowScript"
. ./deepScript.sh

thirdScript.sh

#!/bin/zsh
function _process {
  while read input; do
    echo $input | tr "[a-z]" "[A-Z]"
  done
}
. ./shallowScript.sh | _process

Answer 1

I suppose the problem is that you don't see the prompt after executing the script:

$ ./shallowScript.sh
$ HELLO - THIS IS SHALLOWSCRIPT
HELLO - THIS IS DEEPSCRIPT
(nothing here)

and think it hangs here and waits for the newline. Actually it does not, and the behavior is very expected.

Instead of the newline you can enter any shell command e.g. ls and it will be executed.

$ ./shallowScript.sh
$ HELLO - THIS IS SHALLOWSCRIPT  <--- note the prompt in this line
HELLO - THIS IS DEEPSCRIPT
echo test                        <--- my input
test                             <--- its result
$

What happens here is: the first shell (the one which is running shallowScript.sh) creates a pipe, executes a dup2 call to forward its stdout (fd 1) to the write end of the created pipe and then forks a new process (tr) so that everything the parent prints to stdout is sent to the stdin of tr.

What happens next is that the main shell (the one where you type the initial command ./shallowScript.sh) does not have an idea that it should delay printing the next command prompt until the end of tr process. It knows nothing about tr, so it just waits for the shallowScript.sh to execute, then prints a prompt. The tr is still running at that time, that's why its output (two lines) come after the prompt is printed, and you think the shell is waiting for the newline. It is not actually, it is ready for the next command. You can see the printed prompt ($ character or whatever) somewhere before, inside, or after the output of the script, it depends on how fast the tr process finished.

You see such behavior every time your process forks and the child continues to write to its stdout when the parent is already dead.

Long story short, try this:

$ ./shallowScript.sh | cat
HELLO - THIS IS SHALLOWSCRIPT
HELLO - THIS IS DEEPSCRIPT
$

Here the shell will wait for the cat process to finish before printing a next prompt, and the cat will finish only when all its input (e.g. the output from tr) is processed, just as you expect.

Update: found a relevant quote in zsh docs here: http://zsh.sourceforge.net/Doc/Release/Expansion.html#Process-Substitution

There is an additional problem with >(process); when this is attached to an external command, the parent shell does not wait for process to finish and hence an immediately following command cannot rely on the results being complete. The problem and solution are the same as described in the section MULTIOS in Redirection. Hence in a simplified version of the example above:

paste <(cut -f1 file1) <(cut -f3 file2) > >(process)

(note that no MULTIOS are involved), process will be run asynchronously as far as the parent shell is concerned. The workaround is:

{ paste <(cut -f1 file1) <(cut -f3 file2) } > >(process)

In your case it will give something like this:

{
        print "Hello - this is shallowScript"
        . ./deepScript.sh
} 1> >( tr "[a-z]" "[A-Z]" )

which of course works but looks worse than the original.