Writing outputs to log file and console

Question

In Unix shell, I have a env file (env file defines the parameters required for running the user script like log file name and path, redirect outputs and errors to log file, database connection details, etc) which redirects all the outputs (echo messages) and errors to the log file from the executed script using the following code:

exec 1>>${LOG_FILE}
exec 2>>${LOG_FILE}

The env file is executed at the beginning of each script. Due to the above code in env file all the console outputs that might be user outputs or errors are directly output to the log file which is what I actually needed.

But there are some selective user outputs which I want to be displayed in both the console and the log file. But because of the above code I am not able to do so.

I know that if I remove the above code I can get the desired result for this case, but I will have to manually write all other outputs to the log file which is not an easy task.

Is there a way to get the output in both the console and the log file without removing the above codes?

Answer 1

I have found a way to get the desired output. Though it may be somewhat unorthodox way. Anyways here it goes. In the redir.env file I have following code:

#####redir.env#####    
export LOG_FILE=log.txt

      exec 2>>${LOG_FILE}

    function log {
     echo "$1">>${LOG_FILE}
    }

    function message {
     echo "$1"
     echo "$1">>${LOG_FILE}
    }

Then in the actual script I have the following codes:

#!/bin/sh 
. redir.env
echo "Echoed to console only"
log "Written to log file only"
message "To console and log"
echo "This is stderr. Written to log file only" 1>&2

Here echo outputs only to console, log outputs to only log file and message outputs to both the log file and console.

After executing the above script file I have following outputs:

In console

In console
Echoed to console only
To console and log

For the Log file

In Log File Written to log file only
This is stderr. Written to log file only
To console and log

Hope this help.

Answer 2

    #
    #------------------------------------------------------------------------------
    # echo pass params and print them to a log file and terminal
    # with timestamp and $host_name and $0 PID
    # usage:
    # doLog "INFO some info message"
    # doLog "DEBUG some debug message"
    # doLog "WARN some warning message"
    # doLog "ERROR some really ERROR message"
    # doLog "FATAL some really fatal message"
    #------------------------------------------------------------------------------
    doLog(){
        type_of_msg=$(echo $*|cut -d" " -f1)
        msg=$(echo "$*"|cut -d" " -f2-)
        [[ $type_of_msg == DEBUG ]] && [[ $do_print_debug_msgs -ne 1 ]] && return
        [[ $type_of_msg == INFO ]] && type_of_msg="INFO " # one space for aligning
        [[ $type_of_msg == WARN ]] && type_of_msg="WARN " # as well

        # print to the terminal if we have one
        test -t 1 && echo " [$type_of_msg] `date "+%Y.%m.%d-%H:%M:%S %Z"` [$run_unit][@$host_name] [$$] ""$msg"

        # define default log file none specified in cnf file
        test -z $log_file && \
            mkdir -p $product_instance_dir/dat/log/bash && \
                log_file="$product_instance_dir/dat/log/bash/$run_unit.`date "+%Y%m"`.log"
        echo " [$type_of_msg] `date "+%Y.%m.%d-%H:%M:%S %Z"` [$run_unit][@$host_name] [$$] ""$msg" >> $log_file
    }
    #eof func doLog

Answer 3

exec 3>&1 1>>${LOG_FILE} 2>&1

would send stdout and stderr output into the log file, but would also leave you with fd 3 connected to the console, so you can do

echo "Some console message" 1>&3

to write a message just to the console, or

echo "Some console and log file message" | tee /dev/fd/3

to write a message to both the console and the log file - tee sends its output to both its own fd 1 (which here is the LOG_FILE) and the file you told it to write to (which here is fd 3, i.e. the console).

Example:

exec 3>&1 1>>${LOG_FILE} 2>&1

echo "This is stdout"
echo "This is stderr" 1>&2
echo "This is the console (fd 3)" 1>&3
echo "This is both the log and the console" | tee /dev/fd/3

would print

This is the console (fd 3)
This is both the log and the console

on the console and put

This is stdout
This is stderr
This is both the log and the console

into the log file.

Answer 4

I wanted to display logs on stdout and log file along with the timestamp. None of the above answers worked for me. I made use of process substitution and exec command and came up with the following code. Sample logs:

2017-06-21 11:16:41+05:30 Fetching information about files in the directory...

Add following lines at the top of your script:

LOG_FILE=script.log
exec > >(while read -r line; do printf '%s %s\n' "$(date --rfc-3339=seconds)" "$line" | tee -a $LOG_FILE; done)
exec 2> >(while read -r line; do printf '%s %s\n' "$(date --rfc-3339=seconds)" "$line" | tee -a $LOG_FILE; done >&2)

Hope this helps somebody!

Answer 5

Yes, you want to use tee:

tee - read from standard input and write to standard output and files

Just pipe your command to tee and pass the file as an argument, like so:

exec 1 | tee ${LOG_FILE}
exec 2 | tee ${LOG_FILE}

This both prints the output to the STDOUT and writes the same output to a log file. See man tee for more information.

Note that this won't write stderr to the log file, so if you want to combine the two streams then use:

exec 1 2>&1 | tee ${LOG_FILE}

Answer 6

I tried joonty's answer, but I also got the

exec: 1: not found

error. This is what works best for me (confirmed to work in zsh also):

#!/bin/bash
LOG_FILE=/tmp/both.log
exec > >(tee -a ${LOG_FILE} )
exec 2> >(tee -a ${LOG_FILE} >&2)
echo "this is stdout"
chmmm 77 /makeError

The file /tmp/both.log afterwards contains

this is stdout
chmmm command not found 

The /tmp/both.log is appended unless you remove the -a from tee.

Hint: >(...) is a process substitution. It lets the exec to the tee command as if it were a file.