Write large file

Question

I try to write to a large file, but it seems like it does not work for files larger than 2GB. I have tried with boost::iostreams::file_sink. Is this just a limit with the boost stream? Is there some other way I can write a large file on Win64 and win32?

Answer 1

This depends on:

• The file system which you're using. Old file systems like FAT allow only files up to 2GB. Use NTFS.
• Your library must be compiled to allow large files (64bit seek offsets). If they use 32bit ints to seek in file (check the arguments and results of the calls "seek" and "tell"), you can only ever access 2GB (2^31bits, the 32th is the sign +/-)

This might also help: http://www.boost.org/doc/libs/1_37_0/libs/iostreams/doc/faq.html#offsets

Answer 2

In NTFS the only real limit is the size of the volume. If you want to test this out you can create a dummy file using the command-line:
fsutil file createnew [filename] [validdatalength]
Ex:
fsutil file createnew TestFile.bin 65536000000
That should create a 64GB file assuming you have an NTFS volume.

Answer 3

In Win32/64 the ReadFile & WriteFile can write files larger than 4gb. The positioning is done via a 64bit _LARGE_INTEGER value in SetFilePointerEx. Likewise to get the file size you need GetFileSizeEx and not the basic GetFileSize.