I'm trying to save a couple of files to a container file. The files can be modified later which means the container might have to be enlarged. The user should only see this container as a single file in the file system. The application is written in C++ and running on Windows, but the files should be portable to other platforms as well.
The requirements for the container format are:
1. Size: Allow files bigger than 4GB.
2. Portability: It should be feasible to open the container files on other platforms.
3. Performance: When changing a file inside the container, only the "chunks" that represent that specific file may be changed. Especially, there may not be a complete rewrite of that file.
4. Structure: It should provide a directory structure that allows to build a hierarchy of files and directories.
When leaving out requirement 1.) and 2.) compound files (like used by Word and Excel) are a good fit.
Of course, it's possible to design an API that fulfills all the requirements, but I wonder if there is already some open-source solution that tackles the very problem?
Any suggestions?
A google search for "portable structured storage" yielded: pole
Is the HDF5 File Format what you are looking for if you are building an application in C++
To extend J.F. Sebastian's answer,
tar
might be a better format due to it not compressing. It's literally just a container.I'm not certain how any of these archive formats are at being editable though. Have a play with SharpZipLib and find out, I guess =)
Some archive format (zip, bzip2) might do the job.