Current memory usage display is always ~14MB more

Question

I'm currently using the code in this answer, with some slight modifications as suggested in the comments. However, no matter how many objects I allocate in memory, the listed memory usage is always ~14MB more than what task manager lists. Why could this be?

std::stringstream ss;

PROCESS_MEMORY_COUNTERS_EX pmc;
GetProcessMemoryInfo(GetCurrentProcess(), (PROCESS_MEMORY_COUNTERS*)&pmc, sizeof(pmc));
SIZE_T physMemUsedByMe = pmc.WorkingSetSize;

ss << "\nMEM: " << (physMemUsedByMe / 1024 / 1024) << " MB";    

debugText.setString(ss.str());

Results on a normal build:

debugText:

Task Manager:

Resource Monitor:

Results when allocating 10,000 dummy objects:

debugText:

Task Manager:

Resource Monitor:

EDIT:

After using Resource Monitor (perfmon) as the comments suggested, I found that the column for Working Set matches the memory-listing function I'm using. However, I'm still perplexed as to why there's a ~14MB difference between the Working Set column and the Private column (the latter of which is what Task Manager appears to use). Why is this the case?

Answer 1

Task Manager does not use the Win32 API GetProcessMemoryInfo() function. It uses the NT API ZwQueryInformationProcess() function, setting the ProcessInformationClass parameter to ProcessVmCounters.

Beginning in Windows 10, the following structure is defined (in ntddk.h):

typedef struct _VM_COUNTERS_EX2 {
    VM_COUNTERS_EX CountersEx;
    SIZE_T PrivateWorkingSetSize;
    ULONGLONG SharedCommitUsage;
} VM_COUNTERS_EX2, *PVM_COUNTERS_EX2;

Task Manager uses VM_COUNTERS_EX2 similar to the following code:

VM_COUNTERS_EX2 vm;
ZwQueryInformationProcess(hProcess, ProcessVmCounters, &vm, sizeof(vm), 0);

The value it shows for the "Memory(private working set)" column is the vm.PrivateWorkingSetSize field.

It looks like a Win32 API analog for this does not exist at this time. How you can see this:

typedef struct _VM_COUNTERS_EX {
    SIZE_T PeakVirtualSize;
    SIZE_T VirtualSize;// note this !!
    ULONG PageFaultCount;
    SIZE_T PeakWorkingSetSize;
    SIZE_T WorkingSetSize;
    SIZE_T QuotaPeakPagedPoolUsage;
    SIZE_T QuotaPagedPoolUsage;
    SIZE_T QuotaPeakNonPagedPoolUsage;
    SIZE_T QuotaNonPagedPoolUsage;
    SIZE_T PagefileUsage;
    SIZE_T PeakPagefileUsage;
    SIZE_T PrivateUsage;
} VM_COUNTERS_EX;

The VM_COUNTERS_EX base of VM_COUNTERS_EX2 is very near PROCESS_MEMORY_COUNTERS_EX but not exact (there are no [Peak]VirtualSize) members). GetProcessMemoryInfo() internally calls ZwQueryInformationProcess(hProcess, ProcessVmCounters) and then copies the VM_COUNTERS_EX to PROCESS_MEMORY_COUNTERS_EX.

---

In Task Manager on Windows 10, the "Memory(physical memory reserved by individual processes)" column shows PrivateWorkingSet (in increments of 1024 bytes). This same value is shown under the "Details" tab (private working set). For an unknown reason, that value is shown in increments of 1000 bytes, so the real value is always 1.024 times higher.

But you use the "total" working set - WorkingSetSize - which is the sum of the "private" and "shared" working sets (you need the add columns to the Details tab to view this, by default only the private memory is shown). So, there is a constant difference in the result (14 MB) - this is the "shared" working set (usually common DLLs, like ntdll.dll, kerner32.dll, kernelbase.dll, etc). This difference does not change when you allocate memory (10,000 dummy objects). The "private" working set grows, but the "shared" working set remains unchanged (because no new DLLs are loaded/unloaded).

If you want to show memory like Task Manager does, use the PrivateWorkingSetSize member of VM_COUNTERS_EX2 from the NT API. If you cannot use that, then you will have different results than Task Manager.

If you do not like the NT API, or do not understand it, this is not my problem (or maybe somebody can now get the PrivateWorkingSetSize by using some "documented" API?). If Task Manager uses the NT API, this is also not my choice.