Empty a file while in use in linux

Question

I'm trying to empty a file in linux while in use, it's a log file so it is continuosly written. Right now I've used:

echo -n > filename

or

cat /dev/null > filename

but all of this produce an empty file with a newline character (or strange character that I can see as ^@^@^@^@^@^@^@^@^@^@^@^.. on vi) and I have to remove manually with vi and dd the first line and then save.

If I don't use vi adn dd I'm not able to manipulate file with grep but I need an automatic procedure that i can write in a shell script.

Ideas?

Answer 1

I have not a linux shell here to try ir, but have you try this?

echo "" > file

Answer 2

If it's a log file then the proper way to do this is to use logrotate. As you mentioned doing it manually does not work.

Answer 3

What's going on is fairly simple: you are emptying out the file.

Why is it full of ^@s, then, you ask? Well, in a very real sense, it is not. It does not contain those weird characters. It has a "hole".

The program that is writing to the file is writing a file that was opened with O_WRONLY (or perhaps O_RDWR) but not O_APPEND. This program has written, say, 65536 bytes into the file at the point when you empty out the file with cp /dev/null filename or : > filename or some similar command.

Now the program goes to write another chunk of data (say, 4096 or 8192 bytes). Where will that data be written? The answer is: "at the current seek offset on the underlying file descriptor". If the program used O_APPEND the write would be, in effect, preceded by an lseek call that did a "seek to current end-of-file, i.e., current length of file". When you truncate the file that "current end of file" would become zero (the file becoming empty) so the seek would move the write offset to position 0 and the write would go there. But the program did not use O_APPEND, so there is no pre-write "reposition" operation, and the data bytes are written at the current offset (which, again, we've claimed to be 65536 above).

You now have a file that has no data in byte offsets 0 through 65535 inclusive, followed by some data in byte offsets 65536 through 73727 (assuming the write writes 8192 bytes). That "missing" data is the "hole" in the file. When some other program goes to read the file, the OS pretends there is data there: all-zero-byte data.

If the program doing the write operations does not do them on block boundaries, the OS will in fact allocate some extra data (to fit the write into whole blocks) and zero it out. Those zero bytes are not part of the "hole" (they're real zero bytes in the file) but to ordinary programs that do not peek behind the curtain at the Wizard of Oz, the "hole" zero-bytes and the "non-hole" zero bytes are indistinguishable.

What you need to do is to modify the program to use O_APPEND, or to use library routines like syslog that know how to cooperate with log-rotation operations, or perhaps both.

[Edit to add: not sure why this suddenly showed up on the front page and I answered a question from 2011...]

Answer 4

Why not just :>filename?

(: is a bash builtin having the same effect as /bin/true, and both commands don't echo anything)

Proof that it works:

fg@erwin ~ $ du t.txt
4       t.txt
fg@erwin ~ $ :>t.txt
fg@erwin ~ $ du t.txt
0       t.txt

Answer 5

This should be enough to empty a file:

> file

However, the other methods you said you tried should also work. If you're seeing weird characters, then they are being written to the file by something else - most probably whatever process is logging there.

Answer 6

Another way is the following:

cp /dev/null the_file

The advantage of this technique is that it is a single command, so in case it needs sudo access only one sudo call is required.