Implementing bulk record fetching

Question

At the start of my program, I need to read data from a MS Access database (.mdb) into a drop down control. This is done so that whenever the user types in that control, the application can auto-complete.

Anyway, the reading from database took forever so I thought I'd implement bulk row fetching.

This is the code I have:

CString sDsn;
CString sField;
sDsn.Format("ODBC;DRIVER={%s};DSN='';DBQ=%s",sDriver,sFile);
TRY
{
    // Open the database
    database.Open(NULL,false,false,sDsn);

    // Allocate the rowset
    CMultiRowset recset( &database );

    // Build the SQL statement
    SqlString =  "SELECT NAME "
            "FROM INFOTABLE";

    // Set the rowset size. These many rows will be fetched in one bulk operation
    recset.SetRowsetSize(25);

    // Open the rowset
    recset.Open(CRecordset::forwardOnly, SqlString, CRecordset::readOnly | CRecordset::useMultiRowFetch);

    // Loop through each rowset
    while( !recset.IsEOF() )
    {
        int rowsFetched = (int)recset.GetRowsFetched(); // This value is always 1 somehow
        for( int rowCount = 1; rowCount <= rowsFetched; rowCount++ )
        {
            recset.SetRowsetCursorPosition(rowCount);
            recset.GetFieldValue("NAME",sField);
            m_nameDropDown.AddString(sField);
        }

        // Go to next rowset
        recset.MoveNext();
    }

    // Close the database
    database.Close();
}
CATCH(CDBException, e)
{
    // If a database exception occured, show error msg
    AfxMessageBox("Database error: "+e->m_strError);
}
END_CATCH;

MultiRowset.cpp looks like:

#include "stdafx.h"
#include "afxdb.h"
#include "MultiRowset.h"

// Constructor
CMultiRowset::CMultiRowset(CDatabase *pDB)
   : CRecordset(pDB)
{
    m_NameData = NULL;
    m_NameDataLengths = NULL;

    m_nFields = 1;
    CRecordset::CRecordset(pDB);
}

void CMultiRowset::DoBulkFieldExchange(CFieldExchange *pFX)
{
   pFX->SetFieldType(CFieldExchange::outputColumn);
   RFX_Text_Bulk(pFX, _T("[NAME]"), &m_NameData, &m_NameDataLengths, 30);
}

MultiRowset.h looks like:

#if !defined(__MULTIROWSET_H_AD12FD1F_0566_4cb2_AE11_057227A594B8__)
#define __MULTIROWSET_H_AD12FD1F_0566_4cb2_AE11_057227A594B8__

class CMultiRowset : public CRecordset
{
public:
      // Field data members
      LPSTR m_NameData;

      // Pointers for the lengths of the field data
      long* m_NameDataLengths;

      // Constructor
      CMultiRowset(CDatabase *);

      // Methods
      void DoBulkFieldExchange(CFieldExchange *);
};

#endif

And in my database, the INFOTABLE looks like:

NAME    AGE
----    ---
Name1   Age1
Name2   Age2
      .
      .
      .
      .

All I need to do is only read the data from the database. Can someone please tell me what I'm doing wrong? My code right now behaves exactly like a normal fetch. There's no bulk fetching happening.

EDIT:

I just poked around in DBRFX.cpp and found out that RFX_Text_Bulk() initializes my passed m_NameData as new char[nRowsetSize * nMaxLength]!

This means m_NameData is only a character array! I need to fetch multiple names, so wouldn't I need a 2D character array? The strangest thing is, the same RFX_Text_Bulk() initializes my passed m_NDCDataLengths as new long[nRowsetSize]. Why in the world would a character array need an array of lengths?!

Answer 1

According to http://msdn.microsoft.com/en-us/library/77dcbckz.aspx#_core_how_crecordset_supports_bulk_row_fetching you have to open CRecordset with CRecordset::useMultiRowFetch flag before call SetRowsetSize:

To implement bulk row fetching, you must specify the CRecordset::useMultiRowFetch option in the dwOptions parameter of the Open member function. To change the setting for the rowset size, call SetRowsetSize.

Answer 2

Just faced the same problem. You should use in recset.Open() call for dwOptions parameter only CRecordset::useMultiRowFetch, and not CRecordset::readOnly | CRecordset::useMultiRowFetch. Hope this helps someone...

EDIT:- After re-check here is the situation - when using bulk recordset and opening with CRecordset::forwardOnly and CRecordset::readOnly, you must also specify CRecordset::useExtendedFetch in dwOptions. For other types of scrolling, using CRecordset::readOnly | CRecordset::useMultiRowFetch is just fine.

Answer 3

You almost got it right. To fetch the values, I would change your

        for( int rowCount = 1; rowCount <= rowsFetched; rowCount++ )
        {
            recset.SetRowsetCursorPosition(rowCount);
            recset.GetFieldValue("NAME",sField);
            m_nameDropDown.AddString(sField);
        }

by something like this

for( int nPosInRowset = 0; nPosInRowset < rowsFetched; nPosInRowset++ )
{
    //Check if value is null
    if (*(recset.m_NameDataLengths + nPosInRowset) == SQL_NULL_DATA)
        continue;    

    CString csComboString;
    csComboString = (recset.m_NameData + (nPosInRowset * 30)); //Where 30 is the size specified in RFX_Text_Bulk

    m_nameDropDown.AddString(csComboString);
}

EDIT: To fetch more than one row, remove the CRecordset::forwardOnly option

EDIT 2 : You can also keep CRecordset::forwardonly, but add the CRecordset::useExtendedFetch option