Error PInvoking Function

Question

I have the following code as part of my control. SetReaderMode function creates the structure and calls the function explained here, http://msdn.microsoft.com/en-us/library/bb775599(VS.85).aspx

When I run this code, i get the error

Attempted to read or write protected memory. This is often an indication that other memory is corrupt.

I'm not sure what the issue may be. What am I doing wrong?

        <DllImport("Comctl32.dll", EntryPoint:="#383", _
         CallingConvention:=CallingConvention.StdCall)> _
    Private Shared Sub DoReaderMode(prmi As READERMODEINFO)

    End Sub

    <StructLayout(LayoutKind.Sequential)>
    Private Structure READERMODEINFO
        Dim cbSize As UInt32
        Dim hwnd As IntPtr
        Dim fFlags As UInt32
        Dim prc As IntPtr
        Dim pfnScroll As ReaderScrollCallbackDelegate
        Dim fFlags2 As TranslateDispatchCallbackDelegate
        Dim lParam As IntPtr
    End Structure

    Private Sub SetReaderMode()

        Dim Info As New READERMODEINFO
        Info.hwnd = Me.Handle
        Info.fFlags = 0
        Info.prc = IntPtr.Zero
        Info.pfnScroll = New ReaderScrollCallbackDelegate(AddressOf ReaderScrollCallback)
        Info.fFlags2 = New TranslateDispatchCallbackDelegate(AddressOf TranslateDispatchCallback)
        Info.lParam = IntPtr.Zero
        Info.cbSize = Marshal.SizeOf(Info)

        DoReaderMode(Info)

    End Sub


    Private Delegate Function ReaderScrollCallbackDelegate(ByVal prmi As READERMODEINFO, dx As Integer, dy As Integer) As Boolean
    Private Delegate Function TranslateDispatchCallbackDelegate(lpmsg As IntPtr) As Boolean

    <AllowReversePInvokeCalls()>
    Private Function TranslateDispatchCallback(lpmsg As IntPtr) As Boolean
        Return True
    End Function

    <AllowReversePInvokeCalls()>
    Private Function ReaderScrollCallback(ByVal prmi As READERMODEINFO, dx As Int32, dy As Int32) As Boolean
        Return True
    End Function

Answer 1

Is not an easy nut to crack. Assuming the callback are correct in term of signature/calling convention, a problem can be that since the carbage collector collect Info at the end of the function SetReaderMode, the callback address becames invalid. So try to declare Info as a member variable. If the error remain callback signature has something wrong, but as I said, not so easy to see the error at a glance.

Answer 2

I've figured it out. After reviewing the documentation more closely, I've added a ByRef to the DoReaderMode definition and to the ReaderScrollCallback definition, since the arguments where defined as pointers to structures, not just structures. I also added some other code to pass the rectangle in the ReaderModeInfo structure.

Below is the working code. Interestingly, the documentation states that you click to exit ReaderMode, however when testing it looks like you have to hold the button down and release to exit.

    <DllImport("Comctl32.dll", EntryPoint:="#383", _
         CallingConvention:=CallingConvention.StdCall)> _
    Private Shared Sub DoReaderMode(ByRef prmi As READERMODEINFO)

    End Sub

    <StructLayout(LayoutKind.Sequential)>
    Private Structure READERMODEINFO
        Dim cbSize As UInt32
        Dim hwnd As IntPtr
        Dim fFlags As UInt32
        Dim prc As IntPtr
        Dim pfnScroll As ReaderScrollCallbackDelegate
        Dim fFlags2 As TranslateDispatchCallbackDelegate
        Dim lParam As IntPtr
    End Structure


    Private Sub SetReaderMode()

        Dim SetReaderModeInfo As READERMODEINFO

        Dim rect As New Interop.RECT(Me.Width / 2 - 20, Me.Height / 2 - 20, Me.Width / 2 + 20, Me.Height / 2 + 20)

        Dim pnt As IntPtr = Marshal.AllocHGlobal(Marshal.SizeOf(rect))
        Marshal.StructureToPtr(rect, pnt, True)

        SetReaderModeInfo = New READERMODEINFO
        SetReaderModeInfo.hwnd = Me.Handle
        SetReaderModeInfo.fFlags = 1
        SetReaderModeInfo.prc = pnt
        SetReaderModeInfo.pfnScroll = New ReaderScrollCallbackDelegate(AddressOf ReaderScrollCallback)
        SetReaderModeInfo.fFlags2 = New TranslateDispatchCallbackDelegate(AddressOf TranslateDispatchCallback)
        SetReaderModeInfo.lParam = IntPtr.Zero
        SetReaderModeInfo.cbSize = Marshal.SizeOf(SetReaderModeInfo)

        DoReaderMode(SetReaderModeInfo)

        Marshal.FreeHGlobal(pnt)

    End Sub

    Private Delegate Function ReaderScrollCallbackDelegate(ByRef prmi As READERMODEINFO, dx As Integer, dy As Integer) As Boolean

    Private Delegate Function TranslateDispatchCallbackDelegate(ByRef lpmsg As Interop.MSG) As Boolean

    Private Function TranslateDispatchCallback(ByRef lpmsg As Interop.MSG) As Boolean
        Return False
    End Function

    Private Function ReaderScrollCallback(ByRef prmi As READERMODEINFO, dx As Int32, dy As Int32) As Boolean
        Return True
    End Function