Dynamically generated code execute in wrong addres

Question

This is an UWP app. I am having this issue with ARM/Release build.

Variable out points to extra_memory where the memory's protection is changed by protect_readwrite each time before recompilation and by protect_exec each time before executing the recompiled code. Basically extra_memory is where we put recompiled code.

First of all, after new_dynarec_init is called to change extra_memory's protection to read&write, out is given the pointer to extra_memory such that we can use out as the entry point of executing the recompiled code.

new_recompile_block is the function which does the recompilation and is called from the function new_dyna_start.

extern char extra_memory[33554432];
#define BASE_ADDR ((int)(&extra_memory))
void *base_addr;
u_char *out;

void new_dynarec_init()
{
    protect_readwrite();
    base_addr = ((int)(&extra_memory));
    out = (u_char *)base_addr;
}

int new_recompile_block(int addr)
{
    //if(g_cp0_regs[CP0_COUNT_REG]==365117028) tracedebug=1;
    protect_readwrite();

    //the recompiling code here
    ......

    protect_exec();
    return 0;
}

void protect_readwrite()
{
#if NEW_DYNAREC == NEW_DYNAREC_ARM
    PVOID addr = BASE_ADDR;
#else
    PVOID addr = base_addr;
#endif
    BOOL bVirPro = VirtualProtectFromApp(addr, 1 << TARGET_SIZE_2, PAGE_READWRITE, &oldProt);
    oldProt = PAGE_READWRITE;
    if (!bVirPro)
    {
        OutputDebugString("PAGE_READWRITE fail");
    }
}

void protect_exec()
{
#if NEW_DYNAREC == NEW_DYNAREC_ARM
    PVOID addr = BASE_ADDR;
#else
    PVOID addr = base_addr;
#endif
    BOOL bVirPro = VirtualProtectFromApp(addr, 1 << TARGET_SIZE_2, PAGE_EXECUTE, &oldProt);
    oldProt = PAGE_EXECUTE;
    if (!bVirPro)
    {
        OutputDebugString("PAGE_EXECUTE fail");
    }
}

The dynarec can successfully recompile the code. After the code is generated, the PC is passed the start address of the recompiled code block through "mov pc, r4" in function new_dyna_start to execute the recompiled code. Variable out points to the recompiled code block, so r4 stores value of out through "ldr r4, [r1]".

EXTERN  out

;data section

extra_memory      SPACE 33554432
dynarec_local     SPACE 64

;code section

|.outptr_offset|    DCD out-(|.outptr_pic|+8)

new_dyna_start  GLOBAL_FUNCTION_BEGIN
    ldr    r12, =dynarec_local+28
    ldr    r1, |.outptr_offset|
|.outptr_pic|
    add    r1, pc, r1
    mov    r0, #0xa4000000
    stmia  r12, {r4, r5, r6, r7, r8, r9, sl, fp, lr}
    sub    fp, r12, #28
    ldr    r4, [r1]
    add    r0, r0, #0x40
    bl     new_recompile_block
    ldr    r0, [fp, #next_interupt-dynarec_local]
    ldr    r10, [fp, #g_cp0_regs+36-dynarec_local] ; Count 
    str    r0, [fp, #last_count-dynarec_local]
    sub    r10, r10, r0
    mov    pc, r4
    FUNCTION_END

Each time I run the program, the address of functions and variables will dynamically change. I only take one running sample for reference.

Once in my test, during the first pass of recompilation, 0x2c90 bytes of code were generated. The start address of recompiled block was 0x55577000 that variable out points to. The address of new_dyna_start was 0x537c1648. While the new_recompile_block was executed out, an access failure happened on memory address 0xE88C4FF0(0xC0000005: Access violation executing location 0xE88C4FF0). By using "mov pc, r4", 0x55577000 should have been passed to pc to execute the recompiled code block.

Each time I run the app, it crashes on the same address 0xE88C4FF0. I guess it's not related to the recompiled code or the code that does the recompilation.

Before Windows 10 Mobile, execution of ARM code will crash upon returning back from exception or interrupt handler since the T bit of CPSR is forcibly set to 1 before switching to the handler from ARM code. KeContextFromKframes does this according to this post. Whether this has been changed to allow no-issue ARM code in Windows 10 Mobile is till in debate. Since each time the access failure address is unchanged 0xE88C4FF0, I doubt the crash might be related to this.

Would the app call KeContextFromKframes before "bl new_recompile_block"? If that's the case the app would crash upon returning back to new_dyna_start.

Please help.

Answer 1

It seems that |.outptr_offset| DCD out-(|.outptr_pic|+8) doesn't work at all, which causes r1 stores the address of instruction "stmia r12, {r4, r5, r6, r7, r8, r9, sl, fp, lr}".

Remove the definition of |.outptr_offset|. Loading the address of variable out directly into r1 resolves the issue. The working code is here.

new_dyna_start  GLOBAL_FUNCTION_BEGIN
    ldr    r12, =dynarec_local+28
    ldr    r1, =out
    ......