Using Python to Know When a File Has Completely Be

Question

I am using Python to develop an application that does the following:

• Monitors a particular directory and watches for file to be transferred to it. Once the file has finished its transfer, run some external program on the file.

The main issue I have developing this application is knowing when the file has finished transferring. From what I know the file will be transferred via SFTP to a particular directory. How will Python know when the file has finished transferring? I know that I can use the st_size attribute from the object returned by os.stat(fileName) method. Are there more tools that I need to use to accomplish these goals?

Answer 1

I ended up using a combination of watchdog and waited until I can open the file for writing

 #If there is no error when trying to read the file, then it has completely loaded
    try:
        with io.FileIO(fileName, "r+") as fileObj:

            '''
            Deal with case where FTP client uses extensions such as ".part" and '.filepart" for part of the incomplete downloaded file.
            To do this, make sure file exists before adding it to list of completedFiles.
            '''
            if(os.path.isfile(fileName)):
                completedFiles.append(fileName)
                print "File=" + fileName + " has completely loaded."
    except IOError as ioe:
        print str(ioe)

Answer 2

The best way to solve this would be to have the sending process SFTP to a holding area, and then (presumably using SSH) execute a mv command to move the file from the holding area to the final destination area. Then, once the file appears in the destination area, your script knows that it is completely transferred.