Filesystem watcher and large files

Question

var fsw = new FileSystemWatcher(sPath, "*.PPF");
fsw.NotifyFilter = NotifyFilters.FileName;
fsw.IncludeSubdirectories = true;
fsw.Created += FswCreated;
fsw.EnableRaisingEvents = true;

static void FswCreated(object sender, FileSystemEventArgs e)
{
  string sFile = e.FullPath;
  string[] arrLines = File.ReadAllLines(sFile);
}

this fails with large files, because the process is not finnished with writing the file. The file is copied via the Network, so i dont know the size of the file. What kind of syncronisation is required to make this robust?

Answer 1

Use the DelayedFileSystemWatcher.cs Class http://blogs.msdn.com/b/ahamza/archive/2006/02/06/526222.aspx

and then this code. Check the PrintFileSystemEventHandler eventhandler. It tries to read the file in file stream and if any ioerror is giventhe it assumes the file is still reading so it waits for a interval(2 seconds in this example) and then tries again. Check the CONVERSION: label

static void Main(string[] args)
    {
        DelayedFileSystemWatcher dfw = new DelayedFileSystemWatcher(@"C:\\Test", "*.*");
        dfw.Changed += new FileSystemEventHandler(Program.FileSystemEventHandlerMethod);
        dfw.Created += new FileSystemEventHandler(Program.FileSystemEventHandlerMethod);
        dfw.Deleted += new FileSystemEventHandler(Program.FileSystemEventHandlerMethod);
        dfw.Error += new ErrorEventHandler(Program.ErrorEventHandlerMethod);
        dfw.Renamed += new RenamedEventHandler(Program.RenamedEventHandlerMethod);

        dfw.IncludeSubdirectories = true;
        dfw.ConsolidationInterval = 1000;
        dfw.EnableRaisingEvents = true;
        Console.WriteLine("Press \'q\' to quit the sample.");
        while (Console.Read() != 'q') ;
        //System.Threading.Thread.Sleep(60000);
        dfw.Dispose();
    }
    private static void FileSystemEventHandlerMethod(object sender, FileSystemEventArgs e)
    {
        PrintFileSystemEventHandler(e);
        System.Console.WriteLine();
    }

    private static void ErrorEventHandlerMethod(object sender, ErrorEventArgs e)
    {
        System.Console.WriteLine(e.GetException().Message);
        System.Console.WriteLine();
    }

    private static void RenamedEventHandlerMethod(object sender, RenamedEventArgs e)
    {
        PrintRenamedEventHandler(e);
        System.Console.WriteLine();
    }

    private static void PrintFileSystemEventHandler(FileSystemEventArgs e)
    {

    CONVERSION:
        try
        {

            if (e.ChangeType != WatcherChangeTypes.Deleted)
            {
                if (!isFolder(e.FullPath) && isFile(e.FullPath))
                {
                    FileStream fs = new FileStream(e.FullPath, FileMode.Open, FileAccess.Read, FileShare.None);
                    fs.Close();
                }

            }
            System.Console.WriteLine(e.Name + " " + e.FullPath + " " + e.ChangeType);

        }
        catch (System.IO.IOException)
        {
            Console.WriteLine("There was an IOException error or File is still copying. Retrying in 2 seconds...");
            System.Threading.Thread.Sleep(2000);
            goto CONVERSION;
        }



        //System.Console.WriteLine(e.Name);
        //System.Console.WriteLine(e.FullPath);
        //System.Console.WriteLine(e.ChangeType);
    }

    private static bool isFolder(string strPath)
    {

        bool isFolderExist = false;
        try
        {
            isFolderExist = Directory.Exists(strPath);
        }
        catch
        {
            isFolderExist = false;
        }
        return isFolderExist;
    }

    private static bool isFile(string strPath)
    {

        bool isFileExist = false;
        try
        {
            isFileExist = File.Exists(strPath);
        }
        catch
        {
            isFileExist = false;
        }
        return isFileExist;
    }


    private static void PrintRenamedEventHandler(RenamedEventArgs e)
    {
        PrintFileSystemEventHandler(e);
        System.Console.WriteLine(e.OldName);
        System.Console.WriteLine(e.OldFullPath);
    }

I dont have the link to the project. but this wil help.

Answer 2

Simply in your fswCreated, sleep for about 1/2 seconds with Thread.Sleep(500) if thats possible. That should give you the time the computer needs to finish writing the file.

Of course, for slower hard drives, this may or may enough time.

Answer 3

I support the solution accepted by Shay Erlichmen. But however a) You may wan't to open the File with access mode FileAccess.Read, incase its a read only file

b) Some programs whilst downloading, the file will have some funny extension and when completed, the extension will change, and although file would have completed you will have file not found exception.

So handle the exceptions, and also subscribe to file.renamed event

Answer 4

Solution found on stackoverflow and modified it a bit.

static bool IsFileLocked(FileInfo file)
{
    FileStream stream = null;

    try
    {
        stream = file.Open(FileMode.Open, 
                 FileAccess.ReadWrite, FileShare.None);
    }
    catch (IOException)
    {
        //the file is unavailable because it is:
        //still being written to
        //or being processed by another thread
        //or does not exist (has already been processed)
        return true;
    }
    finally
    {
        if (stream != null)
            stream.Close();
    }

    //file is not locked
    return false;
}

static void FswCreated(object sender, FileSystemEventArgs e)
{
    string sFile = e.FullPath;

    Console.WriteLine("processing file : " + sFile);

    // Wait if file is still open
    FileInfo fileInfo = new FileInfo(sFile);
    while(IsFileLocked(fileInfo))
    {
        Thread.Sleep(500);
    }

    string[] arrLines = File.ReadAllLines(sFile);
}

Answer 5

AFAIK you don't get notified once the copy is done, you can implement a retry mechanism.
If you get a shearing violation just trigger a timer to retry the operation in X seconds.
The second retry should be after X*2 seconds and so on (with some limitation of course).