FTP Binary x ASCII automatic selection

Question

I program a FTP addon (using python's ftplib) and I wonder what would be the best way to determine, if it should use Binary or ASCII transfer method for a given file. In other words "automatic" transfer mode.

I started looking into this as I got issue report about double lines and googling turned out that binary mode should be responsible [1][2].

It might be using whitelisting or blacklisting extension supposed to be binary or ascii, but it might not be present and of course a question which of the methods (wl x bl,b x a) to use then. It does not sound solid way enough (imperfect list, lack of extension).

-> question: For a given file what's a solid way to choose either binary or ascii transfer mode? (if there is any...)
Thank you

[1] http://www.seo-creative.co.uk/tutorials/ftp-client-giving-you-double-line-breaks/
[2] http://www.chami.com/html-kit/support/docs/pages/h000114.html

Answer 1

First of all, you need to inspect the file for BOM marks which identify UTF16 or UTF8. If they are present, than with 99,9% probability the file is a text file.

Next you can scan the file and check if it contains characters with codes less than 32 and not 9, 10 and 13 (TAB, LF, CR). If there are no such characters in the file, then the file is probably a text. But there's no guarantee here OR the user can explicitly request the file to be transferred in binary mode.