How to write a 2D vector into a binary file?

问题:

everyone! I have a 2D vector filled with unsigned chars. Now I want to save its contents into a binary file:

std::vector<std::vector<unsigned char> > v2D(1920, std::vector<unsigned char>(1080));

// Populate the 2D vector here
.....

FILE* fpOut;

// Open for write
if ( (err  = fopen_s( &fpOut, "e:\\test.dat", "wb")) !=0 )
{   
   return;
}

// Write the composite file
size_t nCount = 1920 * 1080 * sizeof(unsigned char);
int nWritten = fwrite((char *)&v2D[0][0], sizeof(unsigned char), nCount, fpOut);

// Close file
fclose(fpOut);

But, when I read test.dat, fill in a new 2D vector, and compare its entries with old ones. I find that the written contents are not the same as the original. Why? What wrong with my write statement? Would you please tell me how to write a 2D vector into a binary file in a right way? Thank you very much!

    #define LON_DATA_ROWS 1920
    #define LON_DATA_COLS 1080

    std::vector<std::vector<float> > m_fLon2DArray(LON_DATA_ROWS, std::vector<float>(LON_DATA_COLS));

    std::ifstream InputFile;

    int nSizeOfLonData = TOTAL_LON_ELEMENTS * sizeof(float);

    std::vector<char> vLonDataBuffer(nSizeOfLonData);

    // Open the file
    InputFile.open(m_sNorminalLonLatFile.c_str(), ios::binary);

    // Unable to open file pszDataFile for reading
    if ( InputFile.fail() )
       return false;

    // Read longitude data buffer
    InputFile.read(&vLonDataBuffer[0], nSizeOfLonData);

    // Close the file object
    InputFile.close();

    // Populate the longitude 2D vector
    for (unsigned i = 0; i < LON_DATA_ROWS; i++) 
    {   
        memcpy(&m_fLon2DArray[i][0], &vLonDataBuffer[(i * LON_DATA_COLS) * sizeof(float)], LON_DATA_COLS * sizeof(float));
    }

    // Some operation put here

    // Write the results to a binary file

回答1:

That is wrong. The data contained by v2D is NOT in contiguous memory. However, each element of v2D (which is a vector) is in contiguous memory. That is, the data contained by v2D[i] is in contiguous memory.

So you should do this:

int nWritten = 0;
for(size_t i = 0; i < v2D.size(); i++ )
{
   if ( v2D[i].size() > 0 )
     nWritten += fwrite(&v2D[i][0], sizeof(unsigned char), v2D[i].size(), fpOut);
}

Or you can use C++ IOStream as:

std::ofstream file("E:\\test.data", std::ofstream::binary);
for(size_t i = 0; i < v2D.size(); i++ )
{
    if ( v2D[i].size() > 0 )
    {
       const char* buffer = static_cast<const char*>(&v2D[i][0]);
       file.write(buffer, v2D[i].size());
    }
}

回答2:

A vector of vectors in memory do not look like this:

|---------|---------|---------|---------|...   (where "-" is one element)
     0         1         2         3        

But rather like this:

|->0x48267a70|->0x894753b2|->0xb8cc92f0|->0x22d8cc71|...

(each vector has a pointer to its internal data which will be reallocated when the growth is over the amount of the already allocated memory).

What you did was you took the address of the internal buffer of the first vector and overindexed it after exhausting its elements. That is undefined behaviour and that's why you got memory junk in your file (but you had an equal chance of crashing the program).

You can write the stored vectors one by one to the file by iterating. In a more idiomatic C++ style:

std::ofstream outfile("e:\\test.dat", std::ios::binary | std::ios::out);
if (!outfile.is_open()) ... // handle error

for (std::vector<std::vector<unsigned char> >::iterator it = v2D.begin();
     it != v2D.end();
     ++it)
{
    outfile.write(&it->front(), it->size() * sizeof(unsigned char));
}

Or, in C++11 the above will look like this:

for (auto& it : v2D) 
{
    outfile.write(&it.front(), it.size() * sizeof(unsigned char)); 
}

In both cases, ios_base::failure exceptions should be handled.

回答3:

There are two major ways of doing 2d matrix: by defining a n*m memory area or by having a list of n pointers to m vectors. you have a variation of the latter.

what you're doing is writing a list of vectors data to the file... Your data is not contiguous, so you're writing the "vector" class instance data to the file.

You can write each of the sub vectors one by one, or to use 1920 * 1080 single vector, and write to file it just as you did.