Calling methods in third-party DLLs

Question

I'm using C# and P-Invoke to get access to objects in the Qt framework (http://qt.digia.com/). I don't seem to be having trouble using functions that return simple types (or void), but whenever I try to use a function that returns an object, the application crashes.

For example, in QtXml4.dll, there is a method QXmlInputSource::data(void) that returns an object of type QString. Here is my wrapper class:

public class QXmlInputSource
{
    // PInvoke - class QString QXmlInputSource::data(void)
    [DllImport("QtXml4.dll", CharSet = CharSet.Unicode, EntryPoint = "?data@QXmlInputSource@@UBE?AVQString@@XZ",
        SetLastError = true, CallingConvention = CallingConvention.ThisCall)]
    static extern IntPtr data(ref IntPtr Ptr);

    private IntPtr mPtr;

    public QXmlInputSource(IntPtr Ptr)
    {
        mPtr = Ptr;
    }

    public override string ToString()
    {
        IntPtr mData = data(ref mPtr);
        return "Epic Fail";
    }

}

And here is some code that hooks (using EasyHook) into a function call that provides a valid QXmlInputSource object:

    // just use a P-Invoke implementation to get native API access from C# (this step is not necessary for C++.NET)
    [DllImport("QtXml4.dll", CharSet = CharSet.Unicode, EntryPoint = "?parse@QXmlSimpleReader@@UAE_NPBVQXmlInputSource@@@Z",
        SetLastError = true, CallingConvention = CallingConvention.ThisCall)]
    [return: MarshalAs(UnmanagedType.Bool)]
    static extern bool XmlParse(IntPtr Reader, IntPtr Source);

    // Intercept all calls to parse XML
    public bool XmlParse_Intercepted(IntPtr Reader, IntPtr Source)
    {
        QXmlInputSource XmlSource = new QXmlInputSource(Source);
        String s = XmlSource.ToString();

        // call original API...
        return XmlParse(Reader, Source);
    }

The hooking code seems to work fine. The Qt application crashes when I make the call to the data() function in my wrapper class. As I said above, the Qt-based application seems to crash whenever the function call returns an object rather than a simple type.

I've tried various combinations of CallingConventions, return types, Marshaling, etc. but haven't stumbled on anything that actually works.

Any help much appreciated.

Also a general thank you to all the contributors on the site - it's an invaluable resource!

Answer 1

You cannot possibly hope to call a C++ library like this using P/invoke. You've simply got the wrong tool for the job.

What you need to do is use a C++/CLI mixed mode layer to do the work. Not only will this have the obvious benefit of actually working, it will be far easier too. Write C++ code that calls the native Qt DLLs. Then expose that code to your C# using managed classes. Finally you can simply add a reference to the C++/CLI library from your C# code and it's all good.