How to truncate a file down to certain size but ke

Question

I have a text file that's appended to over time and periodically I want to truncate it down to a certain size, e.g. 10MB, but keeping the last 10MB rather than the first.

Is there any clever way to do this? I'm guessing I should seek to the right point, read from there into a new file, delete old file and rename new file to old name. Any better ideas or example code? Ideally I wouldn't read the whole file into memory because the file could be big.

Please no suggestions on using Log4Net etc.

Answer 1

You don't need to read the whole file before writing it, especially not if you're writing into a different file. You can work in chunks; reading a bit, writing, reading a bit more, writing again. In fact, that's how all I/O is done anyway. Especially with larger files you never really want to read them in all at once.

But what you propose is the only way of removing data from the beginning of a file. You have to rewrite it. Raymond Chen has a blog post on exactly that topic, too.

Answer 2

If you're okay with just reading the last 10MB into memory, this should work:

using(MemoryStream ms = new MemoryStream(10 * 1024 * 1024)) {
    using(FileStream s = new FileStream("yourFile.txt", FileMode.Open, FileAccess.ReadWrite)) {
        s.Seek(-10 * 1024 * 1024, SeekOrigin.End);
        s.CopyTo(ms);
        s.SetLength(10 * 1024 * 1024);
        s.Position = 0;
        ms.Position = 0; // Begin from the start of the memory stream
        ms.CopyTo(s);
    }
}

Answer 3

You could read the file into a binary stream and use the seek method to just retrieve the last 10MB of data and load them into memory. Then you save this stream into a new file and delete the old one. The text data could be truncated though so you have to decide if this is exceptable.

Look here for an example of the Seek method:

http://www.dotnetperls.com/seek

Answer 4

I tested the solution from "false" but it doesn't work for me, it trims the file but keeps the beginning of it, not the end.

I suspect that CopyTo copies the whole stream instead of starting for the stream position. Here's how I made it work:

int trimSize = 10 * 1024 * 1024;
using (MemoryStream ms = new MemoryStream(trimSize))
{
    using (FileStream s = new FileStream(logFilename, FileMode.Open, FileAccess.ReadWrite))
    {
        s.Seek(-trimSize, SeekOrigin.End);

        byte[] bytes = new byte[trimSize];
        s.Read(bytes, 0, trimSize);
        ms.Write(bytes, 0, trimSize);

        ms.Position = 0;
        s.SetLength(trimSize);
        s.Position = 0;
        ms.CopyTo(s);
    }
}