Python stops reading file using read

Question

I'm trying to read a binary file and am getting confusing results.

f = open('foo.dat','r')

data = f.read()

print len(data), f.tell()

The output is:

61, 600

What is going on here? Only the first 61 bytes are read, but the file object is telling me that I'm at the end of the file (the file is 600 bytes long). What happened to the rest of the file?

I just tried reading it in Matlab and it read it in fine so I'm pretty sure the data file is ok.

The documentation mentions something about blocking: "Notice that when in non-blocking mode, less data than what was requested may be returned, even if no size parameter was given." Am I in non-blocking mode? Seems like that shouldn't matter for a file. How do I switch to blocking mode?

UPDATE @John Machin - Yup! Thank you! Looks like that is indeed what was going on. Here's the output:

600, 600
'm\x1aN\x16\x8d\x1e\x96\x10h\x1a'

The '\x1a' is definitely in there.

Answer 1

It's probably on Windows and you have a Ctrl-Z (the CP/M end-of-file marker, which was inherited by Windows via MS-DOS). Do this:

f = open('foo.dat','rb') # NOTE b for binary
data = f.read() 
print len(data), f.tell() 
print repr(data[60:70])

and show us the output. Ctrl-Z is '\x1a' aka chr(26).

Answer 2

data is a str object, thus len(data) tells you about the length of this string, not the file length in bytes.