I have a set of structures similar to:
typedef struct {
int a;
int b;
} ITEM;
typedef struct {
int orderID;
std::vector<ITEM> items;
} ORDER;
typedef struct {
int orderSetID;
std::vector<ORDER> Orders;
} ORDER_SET;
The problem is that the number of orders ranges between 100,000 to 10,000,000 and the number of ITEMS
in an ORDER
ranges from 1 to 500.
The issue is, as I build the ORDER_SET
, I don't know how many ORDERs there will be. I do know when I add an ORDER
how many ITEMS there will be.
Here are some problems:
1) Ideally once I allocate the memory for all of the ORDERs using Orders.resize(), I'd be able to reuse the memory, but it appears that Orders.clear()
does delete it all.
2) I start with a somewhat reasonable size for Orders, say Orders.resize(500,000)
, but the problem is that when I hit large numbers of orders the resize of the Orders vector takes forever because it has to copy all of the ORDER.item
vectors.
Seems like a similar issue to storing spare matrixes, the issue being that I also don't know how large the structure is going to be before I create it.
Additional information:
1) Using Visual Studio 2008
2) As posted in the comments below I was able to improve the construction of an ORDER_SET
which contains 10000000 Orders in a reasonable time by replacing items
with two fields leaving:
typedef struct {
int orderID;
ITEM singleItem;
std::vector<ITEM> *pItems;
} ORDER;
I put all of the pItems into a separate vector which I use for deleting later.
Now the big issue left is that it appears that calling ORDER_SET.Orders.clear()
takes quite some time. I'm not sure why though.
Is there a call like clear() which doesn't release the memory, but just sets end
to begin
? Once the vector has gotten that large, there really isn't any reason to free the memory, since I might need it again.