I am compiling a c++ static library in vs2008, and in the solution i also have a startup project that uses the lib, and that works fine.
But when using the lib in another solution i get an run-time check failure.
"The value of ESP was not properly saved across a functioncall"
Stepping through the code i noticed a function foo() jumping to bar() instead right before the crash. The functions in question are just regular functions and no function pointers.
Anyone has any clue what might be going on, and why it works when using the lib's from the same solution?
edit: the functions (methods) are part of a class, if that helps.
Forgive me for stating the bleeding obvious here, but... I've seen this sort of thing happen many times before when object (.o) and header (.h) files get out of sync. Especially with respect to virtual methods.
Consider: The object file is compiled with header:
class Foo { virtual void f(); };
But then the header gets changed to:
class Foo { virtual void g(); virtual void f(); };
And for the next object file, the compiler's assumptions about where f() is located in the class's vtable are incorrect.
Oftentimes simply recompiling the world (everything!) will help.
This is most probably due to incompatible calling conventions, where the library and the caller have different ideas about stack layout.
Take a look at MSDN for more info.
Make sure you're compiling in Debug mode and not Release mode. If you attempt to debug a program in Release mode, the data you get back from the debugger will be garbage due to optimizations.
I remember seeing such things when the struct member alignment (/Zp compiler switch) of the binaries was different. You could check that, too.
Setting it via #pragma pack rather than via project settings should be safer.