I have a 64bit program that works with VirtualBox COM interface and implements a frontend for the virtual machine. Recently I started getting weird An invalid or unaligned stack was encountered during an unwind operation exceptions and I wanted to at least understand the causes of this. As I understand the stack needs to be 16byte aligned so, I presume unaligned stack pointer likely to cause this. But the thing is, since all my program does is implement a couple of COM interfaces using the STDMETHOD macros from ATL that should use the correct calling convention then how could I mess up the stack?
Here is an example of the call stack when the issue occurs:
ntdll.dll!00007ffe679ac0b4() Unknown
ntdll.dll!00007ffe67913356() Unknown
msvcrt.dll!__longjmp_internal() Unknown
> VBoxREM.dll!000000006fb0f3c4() Unknown
I tried to goole the __longjmp_internal symbol but did not find anything useful - does it indicate that exception unwind is in progress?
Any pointer on how to approach debugging of this issue or comments what could mess up the stack alignment are welcome, since I understand that in this case it will be impossible to give an exact solution because VirtualBox is involved.
I've faced this baffling problem recently.
I know it only started happening after I switched from the static C/C++ runtime to the DLL version, so that probably means the static version didn't do stack unwinding.
I then traced the assembly code for longjmp() and noticed one of the 1st conditional branches was on _JUMP_BUFFER.Frame.
if it's 0, then restore a bunch of registers and return.
Aha! so that must mean if _JUMP_BUFFER.Frame = 0, unwinding is disabled. I tried it and indeed, problem solved.
I then tried to observe what Frame should be when a setjmp()/longjmp() pair succeeds. I found usually, frame = stack pointer, but when the unwinding fails, frame != SP. So I tried setting Frame to SP and that also eliminates the exception.
I don't know why that works. I know in the SYSV x86-64 ABI, the frame pointer is optional. Maybe setjmp() needs a proper frame pointer and isn't getting one?
Not sure if this helps, but i encountered similar problems (without VMs) with longjmp in combination with x64 Windows.
Turned out that any kind of aligned stack-data (aligned with >= 32byte) in the same scope like the longjmp causes longjmp to come up with the 0xC0000028 when compiled for x64.
#include <setjmpex.h>
void doThe_0xC0000028 ( )
{
jmp_buf jp;
if (!setjmp (jp))
{
// do some stuff ...
// ... then "revert" with longjmp.
longjmp (jp, 1);
}
// having any aligned data on stack (align > 16) in the same scope
// causes longjmp to go: 0xC0000028
//------------------------------------------------------------------
__declspec(align(32)) char buffer[12];
// just accessing buffer somehow - this is apparently needed to generate the faulty 0xC0000028
buffer[0];
}
I reported that as MSVC bug:
https://connect.microsoft.com/VisualStudio/feedback/details/3136150/64bit-longjmp-causing-0xc0000028-with-aligned-stack-data
