How to tail -f the latest log file with a given pa

Question

I work with some log system which creates a log file every hour, like follows:

SoftwareLog.2010-08-01-08
SoftwareLog.2010-08-01-09
SoftwareLog.2010-08-01-10

I'm trying to tail to follow the latest log file giving a pattern (e.g. SoftwareLog*) and I realize there's:

tail -F (tail --follow=name --retry)

but that only follow one specific name - and these have different names by date and hour. I tried something like:

tail --follow=name --retry SoftwareLog*(.om[1])  

but the wildcard statement is resoved before it gets passed to tail and doesn't re-execute everytime tail retries.

Any suggestions?

Answer 1

[Edit: after a quick googling for a tool]

You might want to try out multitail - http://www.vanheusden.com/multitail/

If you want to stick with Dennis Williamson's answer (and I've +1'ed him accordingly) here are the blanks filled in for you.

In your shell, run the following script (or it's zsh equivalent, I whipped this up in bash before I saw the zsh tag):

#!/bin/bash

TARGET_DIR="some/logfiles/"
SYMLINK_FILE="SoftwareLog.latest"
SYMLINK_PATH="$TARGET_DIR/$SYMLINK_FILE"

function getLastModifiedFile {
    echo $(ls -t "$TARGET_DIR" | grep -v "$SYMLINK_FILE" | head -1)
}

function getCurrentlySymlinkedFile {
    if [[ -h $SYMLINK_PATH ]]
    then
        echo $(ls -l $SYMLINK_PATH | awk '{print $NF}')
    else
        echo ""
    fi
}

symlinkedFile=$(getCurrentlySymlinkedFile)
while true
do
    sleep 10
    lastModified=$(getLastModifiedFile)
    if [[ $symlinkedFile != $lastModified ]]
    then
        ln -nsf $lastModified $SYMLINK_PATH
        symlinkedFile=$lastModified
    fi
done

Background that process using the normal method (again, I don't know zsh, so it might be different)...

./updateSymlink.sh 2>&1 > /dev/null

Then tail -F $SYMLINK_PATH so that the tail hands the changing of the symbolic link or a rotation of the file.

This is slightly convoluted, but I don't know of another way to do this with tail. If anyone else knows of a utility that handles this, then let them step forward because I'd love to see it myself too - applications like Jetty by default do logs this way and I always script up a symlinking script run on a cron to compensate for it.

[Edit: Removed an erroneous 'j' from the end of one of the lines. You also had a bad variable name "lastModifiedFile" didn't exist, the proper name that you set is "lastModified"]

Answer 2

#!/bin/bash

PATTERN="$1"

# Try to make sure sub-shells exit when we do.
trap "kill -9 -- -$BASHPID" SIGINT SIGTERM EXIT

PID=0
OLD_FILES=""
while true; do
  FILES="$(echo $PATTERN)"
  if test "$FILES" != "$OLD_FILES"; then
    if test "$PID" != "0"; then
      kill $PID
      PID=0
    fi
    if test "$FILES" != "$PATTERN" || test -f "$PATTERN"; then
      tail --pid=$$ -n 0 -F $PATTERN &
      PID=$!
    fi
  fi
  OLD_FILES="$FILES"
  sleep 1
done

Then run it as: tail.sh 'SoftwareLog*'

The script will lose some log lines if the logs are written to between checks. But at least it's a single script, with no symlinks required.

Answer 3

I haven't tested this, but an approach that may work would be to run a background process that creates and updates a symlink to the latest log file and then you would tail -f (or tail -F) the symlink.

Answer 4

I believe the simplest solution is as follows:

tail -f `ls -tr | tail -n 1`

Now, if your directory contains other log files like "SystemLog" and you only want the latest "SoftwareLog" file, then you would simply include a grep as follows:

tail -f `ls -tr | grep SoftwareLog | tail -n 1`