How to benchmark memory usage of a function?

Question

I notice that Rust's test has a benchmark mode that will measure execution time in ns/iter, but I could not find a way to measure memory usage.

How would I implement such a benchmark? Let us assume for the moment that I only care about heap memory at the moment (though stack usage would also certainly be interesting).

Edit: I found this issue which asks for the exact same thing.

Answer 1

Currently, the only way to get allocation information is the alloc::heap::stats_print(); method (behind #![feature(alloc)]), which calls jemalloc's print_stats().

I'll update this answer with further information once I have learned what the output means.

(Note that I'm not going to accept this answer, so if someone comes up with a better solution...)

Answer 2

With Rust 1.0 and 1.1 you could use the libc crate in order to print the jemalloc statistics:

#![feature(libc)]
extern crate libc;
use libc::*;
extern {fn je_malloc_stats_print (write_cb: extern fn (*const c_void, *const c_char), cbopaque: *const c_void, opts: *const c_char);}
extern fn write_cb (_: *const c_void, message: *const c_char) {
    print! ("{}", String::from_utf8_lossy (unsafe {std::ffi::CStr::from_ptr (message as *const i8) .to_bytes()}));
}

fn main() {
    unsafe {je_malloc_stats_print (write_cb, std::ptr::null(), std::ptr::null())};
}

In later Rust versions (1.8 - 1.14) we have the je_malloc_stats_print renamed to je_stats_print:

#![feature(libc)]
extern crate libc;
extern {fn je_stats_print (write_cb: extern fn (*const libc::c_void, *const libc::c_char), cbopaque: *const libc::c_void, opts: *const libc::c_char);}
extern fn write_cb (_: *const libc::c_void, message: *const libc::c_char) {
    print! ("{}", String::from_utf8_lossy (unsafe {std::ffi::CStr::from_ptr (message as *const i8) .to_bytes()}));}
fn main() {unsafe {je_stats_print (write_cb, std::ptr::null(), std::ptr::null())};}

(playground)

In a single-threaded program that should allow you to get a good measurement of how much memory a structure takes. Just print the statistics before the structure is created and after and calculate the difference.

---

You can also use Valgrind (Massif) to get the heap profile. It works just like with any other C program. Make sure you have debug symbols enabled in the executable (e.g. using debug build or custom Cargo configuration). You can use, say, http://massiftool.sourceforge.net/ to analyse the generated heap profile.

(I verified this to work on Debian Jessie, in a different setting your mileage may vary).

In order to use Rust with Valgrind you'll probably have to switch to the system allocator:

#![feature(alloc_system)]
extern crate alloc_system;

---

jemalloc can be told to dump a memory profile. You can probably do this with the Rust FFI but I haven't investigated this route.

Answer 3

As far as measuring data structure sizes is concerned, this can be done fairly easily through the use of traits and a small compiler plugin. Nicholas Nethercote in his article Measuring data structure sizes: Firefox (C++) vs. Servo (Rust) demonstrates how it works in Servo; it boils down to adding #[derive(HeapSizeOf)] (or occasionally a manual implementation) to each type you care about. This is a good way of allowing precise checking of where memory is going, too; it is, however, comparatively intrusive as it requires changes to be made in the first place, where something like jemalloc’s print_stats() doesn’t. Still, for good and precise measurements, it’s a sound approach.