How to implement 'set -o pipefail' in a PO

Question

I have to implement the BASH set -o pipefail option in a POSIX way so that it works on various LINUX/UNIX flavors. To explain a bit, this option enables the user to verify the successful execution of all piped commands. With this option enabled this command cat app.log | grep 'ERROR' fails if cat fails, otherwise the cat error is suppressed.

So, I found a really nice solution here: http://cfaj.ca/shell/cus-faq-2.html

      run() {
         j=1
         while eval "\${pipestatus_$j+:} false"; do
           unset pipestatus_$j
           j=$(($j+1))
         done
         j=1 com= k=1 l=
         for a; do
           if [ "x$a" = 'x|' ]; then
             com="$com { $l "'3>&-
                         echo "pipestatus_'$j'=$?" >&3
                       } 4>&- |'
             j=$(($j+1)) l=
           else
             l="$l \"\$$k\""
           fi
           k=$(($k+1))
         done
         com="$com $l"' 3>&- >&4 4>&-
                    echo "pipestatus_'$j'=$?"'
         exec 4>&1
         eval "$(exec 3>&1; eval "$com")"
         exec 4>&-
         j=1
         while eval "\${pipestatus_$j+:} false"; do
           eval "[ \$pipestatus_$j -eq 0 ]" || return 1
           j=$(($j+1))
         done
         return 0
       }

The above-mentioned run() function enables the user to invoke the piped commands in such a way:

run cmd1 \| cmd2 \| cmd3

If one of the commands fails you get it in $?

There is a problem however, it does not support the grouping of commands between pipes. I want to be able to invoke something like this:

run echo "test" ; grep "test" \| awk '{print}'

When I do it, the invocation fails. I cannot get the right modification to support the grouping of commands -- the script is a bit too complex for my bash skills...

Could somebody help?

Thanks!

Answer 1

When you type:

run echo "test" ; grep "test" \| awk '{print}'

you invoke run with the arguments echo and "test"; then you invoke grep with arguments "test", |, awk and {print}. Typically, grep is not going to find any of the files called |, awk or {print}.

To invoke run as you wanted, you'd have to escape the semi-colon like you did the | (and you'd need to do things similarly for && or || or & and possibly other components of a command line; the handling of $(...) or backticks `...` needs to be thought about carefully).

If you write:

run echo "test" \; grep "test" \| awk '{print}'

you will at least get all the arguments you intended to run. Whether it then works is debatable; I don't yet understand how the run code you showed is supposed to work.

[...Later...]

It does some fearsome I/O redirections, but wraps each segment of a command separated by a pipe symbol into a separate little packet of hieroglyphs. It assumes that wrapping double quotes around an argument neutralizes it correctly, which is not always true (though it is true a lot of the time).

Answer 2

My two cents:

#!/bin/sh

# Saving the pid of the main shell is required,
# as each element of the pipe is a subshell.
self=$$

lots_and_fail() {
    seq 100
    return 1
}

{ lots_and_fail || kill $self; } | sed s/7/3/

This thing seems to do the job. Thoughts?

Answer 3

Am I missing something?

my-script.sh

#!/bin/sh

set -e

cat app.log | grep 'ERROR' 

printf 'It will never print this.\n'

$ ./my-script.sh
cat: app.log: No such file or directory

Answer 4

The core of your idea should probably involve something like this:

{ cmd1 ; echo $? > status1 ; } | cmd2 && grep -q '^0$' status1 }

In longer form, that would be:

{ cmd1 ; echo $? > status1 ; } |  \
{ cmd2 ; echo $? > status2 ; } |  \
  # ... and so on                 \
  cmdN                         && \
  # ^ note lack of wrapper        \ 
grep -q '^0$' status1 &&          \
grep -q '^0$' status2 &&          \
  # ... and so on, to N-1