My trampoline won't bounce (detouring, C++, GC

Question

It feels like I'm abusing Stackoverflow with all my questions, but it's a Q&A forum after all :) Anyhow, I have been using detours for a while now, but I have yet to implement one of my own (I've used wrappers earlier). Since I want to have complete control over my code (who doesn't?) I have decided to implement a fully functional detour'er on my own, so I can understand every single byte of my code.

The code (below) is as simple as possible, the problem though, is not. I have successfully implemented the detour (i.e a hook to my own function) but I haven't been able to implement the trampoline.

Whenever I call the trampoline, depending on the offset I use, I get either a "segmentation fault" or an "illegal instruction". Both cases ends the same though; 'core dumped'. I think it is because I've mixed up the 'relative address' (note: I'm pretty new to Linux so I have far from mastered GDB).

As commented in the code, depending on sizeof(jmpOp)(at line 66) I either get an illegal instruction or a segmentation fault. I'm sorry if it's something obvious, I'm staying up way too late...

// Header files
#include <stdio.h>
#include <sys/mman.h>
#include <unistd.h>
#include "global.h" // Contains typedefines for byte, ulong, ushort etc...
#include <cstring>

bool ProtectMemory(void * addr, int flags)
{
    // Constant holding the page size value
    const size_t pageSize = sysconf(_SC_PAGE_SIZE);

    // Calculate relative page offset
    size_t temp = (size_t) addr;
    temp -= temp % pageSize;

    // Update address
    addr = (void*) temp;

    // Update memory area protection
    return !mprotect(addr, pageSize, flags);
}

const byte jmpOp[] = { 0xE9, 0x00, 0x00, 0x00, 0x00 };

int Test(void)
{
    printf("This is testing\n");
    return 5;
}

int MyTest(void)
{
    printf("This is ******\n");
    return 9;
}

typedef int (*TestType)(void);

int main(int argc, char * argv[])
{
    // Fetch addresses
    byte * test = (byte*) &Test;
    byte * myTest = (byte*) &MyTest;

    // Call original
    Test();

    // Update memory access for 'test' function
    ProtectMemory((void*) test, PROT_EXEC | PROT_WRITE | PROT_READ);

    // Allocate memory for the trampoline
    byte * trampoline = new byte[sizeof(jmpOp) * 2];

    // Do copy operations
    memcpy(trampoline, test, sizeof(jmpOp));
    memcpy(test, jmpOp, sizeof(jmpOp));

    // Setup trampoline
    trampoline += sizeof(jmpOp);
    *trampoline = 0xE9;

    // I think this address is incorrect, how should I calculate it? With the current
    // status (commented 'sizeof(jmpOp)') the compiler complains about "Illegal Instruction".
    // If I uncomment it, and use either + or -, a segmentation fault will occur...
    *(uint*)(trampoline + 1) = ((uint) test - (uint) trampoline)/* + sizeof(jmpOp)*/;
    trampoline -= sizeof(jmpOp);

    // Make the trampoline executable (and read/write)
    ProtectMemory((void*) trampoline, PROT_EXEC | PROT_WRITE | PROT_READ);

    // Setup detour
    *(uint*)(test + 1) = ((uint) myTest - (uint) test) - sizeof(jmpOp);

    // Call 'detoured' func
    Test();

    // Call trampoline (crashes)
    ((TestType) trampoline)();
    return 0;
}

In case of interest, this is the output during a normal run (with the exact code above):

This is testing
This is **
Illegal instruction (core dumped)

And this is the result if I use +/- sizeof(jmpOp) at line 66:

This is testing
This is ******
Segmentation fault (core dumped)

NOTE: I'm running Ubuntu 32 bit and compile with g++ global.cpp main.cpp -o main -Iinclude

Answer 1

You're not going to be able to indiscriminately copy the first 5 bytes of Test() into your trampoline, followed by a jump to the 6th instruction byte of Test(), because you don't know if the first 5 bytes comprise an integral number of x86 variable-length instructions. To do this, you're going to have to do at least a minimal amount of automated disassembling of the Test() function in order to find an instruction boundary that's 5 or more bytes past the beginning of the function, then copy an appropriate number of bytes to your trampoline, and THEN append your jump (which won't be at a fixed offset within your trampoline). Note that on a typical RISC processor (like PPC), you wouldn't have this problem, as all instructions are the same width.