Can I get the NUMA node from a pointer address (in

Question

I've set up my code to carefully load and process data locally on my NUMA system. I think. That is, for debugging purposes I'd really like to be able to use the pointer addresses being accessed inside a particular function, which have been set up by many other functions, to directly identify the NUMA node(s) that the memory pointed at is residing on, so I can check that everything is located where it should be located. Is this possible?

I found this request on msdn http://social.msdn.microsoft.com/Forums/en-US/parallelcppnative/thread/37a02e17-e160-48d9-8625-871ff6b21f72 for the same thing, but the answer uses QueryWorkingSetEx() which appears to be Windows specific. Can this be done on Linux? I'm on Debian Squeeze, to be precise.

Thanks.

Answer 1

There is an move_pages function in -lnuma: http://linux.die.net/man/2/move_pages

which can report current state of address(page) to node mappings:

nodes can also be NULL, in which case move_pages() does not move any pages but instead will return the node where each page currently resides, in the status array. Obtaining the status of each page may be necessary to determine pages that need to be moved.

So, call may be like:

 void * ptr_to_check = your_address;
 /*here you should align ptr_to_check to page boundary */
 int status[1];
 int ret_code;
 status[0]=-1;
 ret_code=move_pages(0 /*self memory */, 1, &ptr_to_check,
    NULL, status, 0);
 printf("Memory at %p is at %d node (retcode %d)\n", ptr_to_check, status[0], ret_code);

Answer 2

Alternatively, there is a get_mempolicy function in -lnuma: http://linux.die.net/man/2/get_mempolicy

If flags specifies both MPOL_F_NODE and MPOL_F_ADDR, get_mempolicy() will 
return the node ID of the node on which the address addr is allocated into 
the location pointed to by mode. If no page has yet been allocated for the 
specified address, get_mempolicy() will allocate a page as if the process had 
performed a read [load] access to that address, and return the ID of the node 
where that page was allocated. 

Thus, the numa node of a page being pointed at by ptr is checked with:

int numa_node = -1;
get_mempolicy(&numa_node, NULL, 0, (void*)ptr, MPOL_F_NODE | MPOL_F_ADDR);
return numa_node;