redirect COPY of stdout to log file from within ba

Question

I know how to redirect stdout to a file:

exec > foo.log
echo test

this will put the 'test' into the foo.log file.

Now I want to redirect the output into the log file AND keep it on stdout

i.e. it can be done trivially from outside the script:

script | tee foo.log

but I want to do declare it within the script itself

I tried

exec | tee foo.log

but it didn't work.

Answer 1

Bash 4 has a coproc command which establishes a named pipe to a command and allows you to communicate through it.

Answer 2

Can't say I'm comfortable with any of the solutions based on exec. I prefer to use tee directly, so I make the script call itself with tee when requested:

# my script: 

check_tee_output()
{
    # copy (append) stdout and stderr to log file if TEE is unset or true
    if [[ -z $TEE || "$TEE" == true ]]; then 
        echo '-------------------------------------------' >> log.txt
        echo '***' $(date) $0 $@ >> log.txt
        TEE=false $0 $@ 2>&1 | tee --append log.txt
        exit $?
    fi 
}

check_tee_output $@

rest of my script

This allows you to do this:

your_script.sh args           # tee 
TEE=true your_script.sh args  # tee 
TEE=false your_script.sh args # don't tee
export TEE=false
your_script.sh args           # tee

You can customize this, e.g. make tee=false the default instead, make TEE hold the log file instead, etc. I guess this solution is similar to jbarlow's, but simpler, maybe mine has limitations that I have not come across yet.

Answer 3

Inside your script file, put all of the commands within parentheses, like this:

(
echo start
ls -l
echo end
) | tee foo.log

Answer 4

#!/usr/bin/env bash

# Redirect stdout ( > ) into a named pipe ( >() ) running "tee"
exec > >(tee -i logfile.txt)

# Without this, only stdout would be captured - i.e. your
# log file would not contain any error messages.
# SEE (and upvote) the answer by Adam Spiers, which keeps STDERR
# as a separate stream - I did not want to steal from him by simply
# adding his answer to mine.
exec 2>&1

echo "foo"
echo "bar" >&2

Note that this is bash, not sh. If you invoke the script with sh myscript.sh, you will get an error along the lines of syntax error near unexpected token '>'.

If you are working with signal traps, you might want to use the tee -i option to avoid disruption of the output if a signal occurs. (Thanks to JamesThomasMoon1979 for the comment.)

---

Tools that change their output depending on whether they write to a pipe or a terminal (ls using colors and columnized output, for example) will detect the above construct as meaning that they output to a pipe.

There are options to enforce the colorizing / columnizing (e.g. ls -C --color=always). Note that this will result in the color codes being written to the logfile as well, making it less readable.

Answer 5

Using the accepted answer my script kept returning exceptionally early (right after 'exec > >(tee ...)') leaving the rest of my script running in the background. As I couldn't get that solution to work my way I found another solution/work around to the problem:

# Logging setup
logfile=mylogfile
mkfifo ${logfile}.pipe
tee < ${logfile}.pipe $logfile &
exec &> ${logfile}.pipe
rm ${logfile}.pipe

# Rest of my script

This makes output from script go from the process, through the pipe into the sub background process of 'tee' that logs everything to disc and to original stdout of the script.

Note that 'exec &>' redirects both stdout and stderr, we could redirect them separately if we like, or change to 'exec >' if we just want stdout.

Even thou the pipe is removed from the file system in the beginning of the script it will continue to function until the processes finishes. We just can't reference it using the file name after the rm-line.

Answer 6

Solution for busybox, macOS bash, and non-bash shells

The accepted answer is certainly the best choice for bash. I'm working in a Busybox environment without access to bash, and it does not understand the exec > >(tee log.txt) syntax. It also does not do exec >$PIPE properly, trying to create an ordinary file with the same name as the named pipe, which fails and hangs.

Hopefully this would be useful to someone else who doesn't have bash.

Also, for anyone using a named pipe, it is safe to rm $PIPE, because that unlinks the pipe from the VFS, but the processes that use it still maintain a reference count on it until they are finished.

Note the use of $* is not necessarily safe.

#!/bin/sh

if [ "$SELF_LOGGING" != "1" ]
then
    # The parent process will enter this branch and set up logging

    # Create a named piped for logging the child's output
    PIPE=tmp.fifo
    mkfifo $PIPE

    # Launch the child process with stdout redirected to the named pipe
    SELF_LOGGING=1 sh $0 $* >$PIPE &

    # Save PID of child process
    PID=$!

    # Launch tee in a separate process
    tee logfile <$PIPE &

    # Unlink $PIPE because the parent process no longer needs it
    rm $PIPE    

    # Wait for child process, which is running the rest of this script
    wait $PID

    # Return the error code from the child process
    exit $?
fi

# The rest of the script goes here