How do I use the Rust memory allocator for a C lib

Question

I'm writing Rust bindings to a C library which has the option to use a third-party memory allocator. Its interface looks like this:

struct allocator {
    void*(*alloc)(void *old, uint);
    void(*free)(void*);
};

The corresponding Rust struct is, I guess, the following:

#[repr(C)]
#[derive(Copy, Clone, Debug, PartialEq)]
pub struct Allocator {
    alloc: Option<extern "C" fn(*mut c_void, c_uint) -> *mut c_void>,
    free: Option<extern "C" fn(*mut c_void)>,
}

How can I implement these two extern functions that should mimic the allocator? I did not find anything really looking like the allocator API in Rust (I understand why however), so I'm curious if it is possible.

Answer 1

It's not as easy as you might like.

The allocation methods are exposed in the heap module of the alloc crate.

Creating some wrapper methods and populating the struct is straight-forward, but we quickly run into an issue:

#![feature(heap_api)]

extern crate libc;
extern crate alloc;

use libc::{c_void, c_uint};
use alloc::heap;

#[repr(C)]
#[derive(Copy, Clone, Debug, PartialEq)]
pub struct Allocator {
    alloc: Option<extern "C" fn(*mut c_void, c_uint) -> *mut c_void>,
    free: Option<extern "C" fn(*mut c_void)>,
}


extern "C" fn alloc_ext(old: *mut c_void, size: c_uint) -> *mut c_void {
    if old.is_null() {
        heap::allocate(size as usize, align) as *mut c_void
    } else {
        heap::reallocate(old as *mut u8, old_size, size as usize, align) as *mut c_void
    }
}

extern "C" fn free_ext(old: *mut c_void) {
    heap::deallocate(old as *mut u8, old_size, align);
}

fn main() {
    Allocator {
        alloc: Some(alloc_ext),
        free: Some(free_ext),
    };
}

The Rust allocator expects to be told the size of any previous allocation as well as the desired alignment. The API you are matching doesn't have any way of passing that along.

Alignment should (I'm not an expert) be OK to hardcode at some value, say 16 bytes. The size is trickier. You will likely need to steal some old C tricks and allocate a little bit extra space to store the size in. You can then store the size and return a pointer just past that.

A completely untested example:

#![feature(alloc, heap_api)]

extern crate libc;
extern crate alloc;

use libc::{c_void, c_uint};
use alloc::heap;
use std::{mem, ptr};

#[repr(C)]
#[derive(Copy, Clone, Debug, PartialEq)]
pub struct Allocator {
    alloc: Option<extern "C" fn(*mut c_void, c_uint) -> *mut c_void>,
    free: Option<extern "C" fn(*mut c_void)>,
}

const ALIGNMENT: usize = 16;

extern "C" fn alloc_ext(old: *mut c_void, size: c_uint) -> *mut c_void {
    unsafe {
        // Should check for integer overflow
        let size_size = mem::size_of::<usize>();
        let size = size as usize + size_size;

        let memory = if old.is_null() {
            heap::allocate(size, ALIGNMENT)
        } else {
            let old = old as *mut u8;
            let old = old.offset(-(size_size as isize));
            let old_size = *(old as *const usize);
            heap::reallocate(old, old_size, size, ALIGNMENT)
        };

        *(memory as *mut usize) = size;
        memory.offset(size_size as isize) as *mut c_void
    }
}

extern "C" fn free_ext(old: *mut c_void) {
    if old.is_null() { return }

    unsafe {
        let size_size = mem::size_of::<usize>();

        let old = old as *mut u8;
        let old = old.offset(-(size_size as isize));
        let old_size = *(old as *const usize);

        heap::deallocate(old as *mut u8, old_size, ALIGNMENT);
    }
}

fn main() {
    Allocator {
        alloc: Some(alloc_ext),
        free: Some(free_ext),
    };

    let pointer = alloc_ext(ptr::null_mut(), 54);
    let pointer = alloc_ext(pointer, 105);
    free_ext(pointer);
}

Isn't [... using Vec as an allocator ...] the more high-level solution?

That's certainly possible, but I'm not completely sure how it would work with reallocation. You'd also have to keep track of the size and capacity of the Vec in order to reconstitute it to reallocate / drop it.