Limit on memory allocation in windows + am I calcu

问题:

I'm writing a program that requires a lot of memory (large graph analysis).

Currently there are two main data structures in my program (taking up most of the memory). These are:

• a n*n matrix of type int **
• and array of length n, type Node *

Node, in this case, is a struct containing two ints (sizeof(Node) = 8)

The biggest value for n that I can run my code on is 22900, doing a bit of calculation I get:

22900*22900 * sizeof(int) * 8 + 22900 * sizeof(Node) = 16782591360 bits

This is 1.95375077 Gigabytes.

So question 1: am I calculating the memory usage for these two data structures properly? and 2: Is there a 2GB memory allocation limit on windows. If so, how can I get around it?

For further information, I am on a 64bit Windows 7 machine compiling with GCC, 4GB RAM with ~3GB of free RAM at time of running.

Thanks.

回答1:

You aren't calculating it correctly. First, there is no reason to multiply anything by 8. The quantum of allocation in C is byte, not bit. Second, you neglect the pointer array which implements the first dimension of your matrix. So:

22900 * sizeof(int*) + 22900*22900*sizeof(int) + 22900*sizeof(Node) = 2097914800 bytes

As for useful advice, I'll leave that to the (already posted) other answer.

回答2:

You are most likely compiling for 32-bits; on windows, 32-bit processes are limited to 2G of addressable space (with a 64-bit OS and the IMAGE_FILE_LARGE_ADDRESS_AWARE flag set, 4GB). Compile for 64-bit and you should see your memory limit rise substantially. However, you will likely want more physical RAM before doing so; you're using half of it already and hitting swap will kill your performance.

回答3:

32bit processes are limited to 2G of user-adressable memory (on most releases of windows with default settings). 64bit processes have much larger address spaces. See this note Performance and Memory Consumption Under WOW64 for a way to give your 32bit app a 4G address space (not sure if/how GCC can build executable images with that flag set though).

Compile your code as a 64bit application and that limit should vanish (try MinGW-w64).

回答4:

To get around the memory limitation, you have to compile the program in 64-bit mode; note that pointers are then 8 byte in size. The total memory usage of the matrix would be then doubled.