Capturing (externally) the memory consumption of a

Question

The Problem

Lets say I have this function:

function hog($i = 1) // uses $i * 0.5 MiB, returns $i * 0.25 MiB
{
    $s = str_repeat('a', $i * 1024 * 512); return substr($s, $i * 1024 * 256);
}

I would like to call it and be able to inspect the maximum amount of memory it uses.

In other words: memory_get_function_peak_usage($callback);. Is this possible?

---

What I Have Tried

I'm using the following values as my non-monotonically increasing $i argument for hog():

$iterations = array_merge(range(0, 50, 10), range(50, 0, 5));
$iterations = array_fill_keys($iterations, 0);

Which is essentially:

(
    [0] => 0
    [10] => 0
    [20] => 0
    [30] => 0
    [40] => 0
    [50] => 0
    [45] => 0
    [35] => 0
    [25] => 0
    [15] => 0
    [5] => 0
)

---

Enclosing with memory_get_usage()

foreach ($iterations as $key => $value)
{
    $alpha = memory_get_usage(); hog($key);
    $iterations[$key] = memory_get_usage() - $alpha;
}

print_r($iterations);

Output:

(
    [0] => 96
    [10] => 0
    [20] => 0
    [30] => 0
    [40] => 0
    [50] => 0
    [45] => 0
    [35] => 0
    [25] => 0
    [15] => 0
    [5] => 0
)

If I store the return value of hog(), the results start to look more realistic:

foreach ($iterations as $key => $value)
{
    $alpha = memory_get_usage(); $s = hog($key);
    $iterations[$key] = memory_get_usage() - $alpha; unset($s);
}

print_r($iterations);

Output:

(
    [0] => 176
    [10] => 2621536
    [20] => 5242976
    [30] => 7864416
    [40] => 10485856
    [50] => 13107296
    [45] => 11796576
    [35] => 9175136
    [25] => 6553696
    [15] => 3932256
    [5] => 1310816
)

As expected, now it's showing me the amount of memory returned, but I need the total memory used.

---

Using register_tick_function():

I didn't knew, but it turns out that when you do:

declare (ticks=1)
{
    $a = hog(1);
}

It won't tick for every line, statement or block of code inside of hog() function, only for the code inside the declare block - so, unless the function is defined within it, this option is a no go.

---

Mixing with gc_* functions:

I tried (without much hope I must say) using combinations of gc_disable(), gc_enable() and gc_collect_cycles() with both experiments above to see if anything changed - it didn't.

Answer 1

I found this: github.com/kampaw/profiler that seem to use the "tick/register/decare-concept", that wasn't an option for you. I've also read that the register_tick_functionality is going to removed in PHP 6. (But that might just be a rumour)

I understand totally what you mean by checking memory INSIDE of the function.

I've tested your code based on just calling the function and then returning memory usage afterward using that function. I made a checkMemoryFunction() just make it more general (of course checkMemoryFunction would take a bit of memory to, but that might be possible to subtract if necessary). I think you were thinking right about getting used memory, BUT I found another very weird thing...

...with this code:

<?php
function hog($i) 
{
    $s = str_repeat('a', $i * 1024 * 512); 
    return substr($s, $i * 1024 * 256);
}

function checkMemoryFunction($functionName, $i) {
    $startMemory = memory_get_usage();
    call_user_func($functionName, $i);
    return memory_get_usage() - $startMemory;
}

$iterations = array_merge(range(0, 50, 10), range(50, 0, 5));
$iterations = array_fill_keys($iterations, 0);

foreach ($iterations as $key => $value)
{
    $mem = checkMemoryFunction('hog', $key);
    $iterations[$key] = $mem;
}

$peak = max($iterations);
echo '<hr />Iteratins array:';
print_r($iterations);
echo 'memory peak=' . $peak;
?>

I got about same results as you did: (First element set, but not the rest)

Output Iterations array:

Array ( [0] => 312 [10] => 0 [20] => 0 [30] => 0 [40] => 0 [50] => 0 [45] => 0 [35] => 0 [25] => 0 [15] => 0 [5] => 0 ) memory peak=312

However, when I add rows to set each key-value to 0 (or whatever value)...

$iterations = array_merge(range(0, 50, 10), range(50, 0, 5));
$iterations = array_fill_keys($iterations, 0);

// set each value to 0 in array
foreach ($iterations as $key => &$value)
{
    $value = 0;
}

foreach ($iterations as $key => $value)
{
    $mem = checkMemoryFunction('hog', $key);
    $iterations[$key] = $mem;
}

...I get these values (some memory usage for all function calls):

Array ( [0] => 312 [10] => 24 [20] => 24 [30] => 24 [40] => 24 [50] => 24 [45] => 24 [35] => 24 [25] => 24 [15] => 24 [5] => 24 ) memory peak=312

So it seems like the problem lied within the call to array_fill_keys(). (It seems like the array-elements aren't initilized in some weird way)

Taking a close look at $iterations-array directly in your code after merging the arrays, it look like this: (duplicate values)

Array ( [0] => 0 [1] => 10 [2] => 20 [3] => 30 [4] => 40 [5] => 50 [6] => 50 [7] => 45 [8] => 40 [9] => 35 [10] => 30 [11] => 25 [12] => 20 [13] => 15 [14] => 10 [15] => 5 [16] => 0` )

but I think what you really wanted was something like this:

Array ( [0] => 0 [1] => 10 [2] => 20 [3] => 30 [4] => 40 [5] => 50 [6] => 45 [8] => 35 [10] => 25 [12] => 15 [14] => 5) 

My suspicion was that the duplicates made array_fill_keys() act in strange manner, so I tried:

$iterations = array(0, 10, 20, 30, 40, 50, 45, 35, 25, 15);
$iterations = array_fill_keys($iterations, 0);
foreach ($iterations as $key => $value)
{
        $mem = checkMemoryFunction('hog', $key);
        $iterations[$key] = $mem;
}

But it still didn't work as expected:

Array ( [0] => 312 [10] => 0 [20] => 0 [30] => 0 [40] => 0 [50] => 0 [45] => 0 [35] => 0 [25] => 0 [15] => 0 ) memory peak=312

When I add

foreach ($iterations as $key => &$value)
{
    $value = 0;
}

again it workds like expected:

Array ( [0] => 312 [10] => 0 [20] => 24 [30] => 24 [40] => 24 [50] => 24 [45] => 32 [35] => 48 [25] => 24 [15] => 24 [5] => 24 ) memory peak=312

I think it's strange because array_fill_keys($iterations, 0); should do the same thing as the foreach above. I can't figure out WHY it doesn't work as expected. Maybe it's a bug, but probably it's something "stupid" that I haven't thought of.

Another approach could be like storing "content inside a function" from theh PHP-source-file and then save it as profile/hog.php and after that execute the hog.php code.

I hope this could help you out a bit! :-)

Answer 2

That has everything to do with the variable scope. Everything inside the function will be cleared once the function ends. So if you need to know how much memory is used in total you need to declare them outside the function scope. My vote would go to a single variable array for ease incase you need multiple vars. If you need just one you obviously dont need the array.

<?php
$outervar = array();

function hog($i = 1) // uses $i * 0.5 MiB, returns $i * 0.25 MiB 
{ 
  global $outervar;
  $outervar['s'] = str_repeat('a', $i * 1024 * 512); return substr($outervar['s'], $i * 1024 * 256); 
}

foreach ($iterations as $key => $value) 
{ 
  $alpha = memory_get_usage(); 
  hog($key); 
  $iterations[$key] = memory_get_usage() - $alpha;
  $outervar = array(); 
 }

print_r($iterations);

Since we dont store the result of hog this will use 0.5mb*$i. If you also need the return value even if its not stored, first save it to $outervar['result'] or something and then return it. But then it will be counted double if you do save it.

A second option would be to gave a second parameter by reference &$memusage and use the memory_get_usage() part inside the function and store the result in the byref variable

Answer 3

I was digging in the PHP manual and I found the memtrack extension, not perfect but it's something.

---

EDIT: I had heard about it, but never actually tried it before. Turns out XHProf is all I needed:

$flags = array
(
    XHPROF_FLAGS_CPU,
    XHPROF_FLAGS_MEMORY,
    XHPROF_FLAGS_NO_BUILTINS,
);

$options = array
(
    'ignored_functions' => array
    (
        'call_user_func',
        'call_user_func_array',
        'xhprof_disable',
    ),
);

function hog($i = 1) // uses $i * 0.5 MiB, returns $i * 0.25 MiB
{
    $s = str_repeat('a', $i * 1024 * 512); return substr($s, $i * 1024 * 256);
}

---

Test #1:

xhprof_enable(array_sum($flags), $options);

hog(4);

$profile = xhprof_disable();

print_r($profile);

Output:

    [main()==>hog] => Array
        (
            [ct] => 1
            [wt] => 54784
            [mu] => 384
            [pmu] => 3142356
        )

    [main()] => Array
        (
            [ct] => 1
            [wt] => 55075
            [mu] => 832
            [pmu] => 3142356
        )

mu is memory usage, pmu is peak memory usage, 3142356 / 1024 / 1024 / 0.5 = 4 = $i.

---

Test #2 (without XHPROF_FLAGS_NO_BUILTINS):

    [hog==>str_repeat] => Array
        (
            [ct] => 1
            [wt] => 21890
            [cpu] => 4000
            [mu] => 2097612
            [pmu] => 2094200
        )

    [hog==>substr] => Array
        (
            [ct] => 1
            [wt] => 17202
            [cpu] => 4000
            [mu] => 1048992
            [pmu] => 1048932
        )

    [main()==>hog] => Array
        (
            [ct] => 1
            [wt] => 45978
            [cpu] => 8000
            [mu] => 1588
            [pmu] => 3143448
        )

    [main()] => Array
        (
            [ct] => 1
            [wt] => 46284
            [cpu] => 8000
            [mu] => 2132
            [pmu] => 3143448
        )

Whoohoo! Thanks Facebook!

---

From the XHProf docs:

It is worth clarifying that that XHProf doesn't strictly track each allocation/free operation. Rather it uses a more simplistic scheme. It tracks the increase/decrease in the amount of memory allocated to PHP between each function's entry and exit. It also tracks increase/decrease in the amount of peak memory allocated to PHP for each function.