Reading multiple precision binary files through fr

Question

I have a huge binary file which has records with multiple precision as {'Double','Double', 'Int32','Int8','Char'}. I have used memmapfile to read in the data but its painfully slow to read in the data. Is there a way to read the whole file through fread?

Answer 1

You can use the 'skip' option of the FREAD function as well as FSEEK to read the records one "column" at-a-time:

%# type and size in byte of the record fields
recordType = {'double' 'double' 'int32' 'int8' 'char'};
recordLen = [8 8 4 1 1];
R = cell(1,numel(recordType));

%# read column-by-column
fid = fopen('file.bin','rb');
for i=1:numel(recordType)
    %# seek to the first field of the first record
    fseek(fid, sum(recordLen(1:i-1)), 'bof');

    %# % read column with specified format, skipping required number of bytes
    R{i} = fread(fid, Inf, ['*' recordType{i}], sum(recordLen)-recordLen(i));
end
fclose(fid);

This code should work for any binary records file in general, you just have to specify the data types and byte length of the records fields. The result will be returned in a cell array containing the columns.