How to 'grep' a continuous stream?

Question

Is that possible to use grep on a continuous stream?

What I mean is sort of a tail -f <file> command, but with grep on the output in order to keep only the lines that interest me.

I've tried tail -f <file> | grep pattern but it seems that grep can only be executed once tail finishes, that is to say never.

Answer 1

Use awk(another great bash utility) instead of grep where you dont have the line buffered option! It will continuously stream your data from tail.

this is how you use grep

tail -f <file> | grep pattern

This is how you would use awk

tail -f <file> | awk '/pattern/{print $0}'

Answer 2

sed would be the proper command (stream editor)

tail -n0 -f <file> | sed -n '/search string/p'

and then if you wanted the tail command to exit once you found a particular string:

tail --pid=$(($BASHPID+1)) -n0 -f <file> | sed -n '/search string/{p; q}'

Obviously a bashism: $BASHPID will be the process id of the tail command. The sed command is next after tail in the pipe, so the sed process id will be $BASHPID+1.

Answer 3

I use the tail -f <file> | grep <pattern> all the time.

It will wait till grep flushes, not till it finishes (I'm using Ubuntu).

Answer 4

In most cases, you can tail -f /var/log/some.log |grep foo and it will work just fine.

If you need to use multiple greps on a running log file and you find that you get no output, you may need to stick the --line-buffered switch into your middle grep(s), like so:

tail -f /var/log/some.log | grep --line-buffered foo | grep bar

Answer 5

Turn on grep's line buffering mode when using BSD grep (FreeBSD, Mac OS X etc.)

tail -f file | grep --line-buffered my_pattern

You don't need to do this for GNU grep (used on pretty much any Linux) as it will flush by default (YMMV for other Unix-likes such as SmartOS, AIX or QNX).