Marshal.Copy not copying from bytes array into str

2019-08-22 21:10发布

问题:

(This is in some sense a follow-up question to Extracting structs in the middle of a file into my structures other than accessing the file byte for byte and compose their value.)

I am having a file containing a structure record in its stream:

[Start]...[StructureRecord]...[End]

The contained structure fits into this layout, of which exists a variable:

Public Structure HeaderStruct
    Public MajorVersion As Short
    Public MinorVersion As Short
    Public Count As Integer         
End Structure

private grHeaderStruct As HeaderStruct

It is into this variable I want a copy of the structure residing in the file.

Required namespaces:

'File, FileMode, FileAccess, FileShare:
Imports System.IO

'GCHandle, GCHandleType:
Imports System.Runtime.InteropServices

'SizeOf, Copy:
Imports System.Runtime.InteropServices.Marshal

My FileStream is named oFS.

Using oFS = File.Open(sFileName, FileMode.Open, FileAccess.Read)
    ...

Assume, that oFS is positioned at the start of [StructureRecord] now.

So there are 8 bytes (HeaderStruct.Length) to read from the file, and copy these into a record instance of this structure. To do so, I wrap the logic to read from file the required number of bytes and transfer them to my structure record into a generic method ExtractStructure. The destination is instantiated just before the call to the routine.

    grHeaderStruct = New HeaderStruct
    ExtractStructure(Of HeaderStruct)(oFS, grHeaderStruct)
    ...
End Using

(Before suggesting techniques to read just these 8 bytes outside of a dedicated method, you should probably know, that the whole file consists of structures, which depend on each other. The Count field says, that 3 child structures are to follow, but these contain Count fields themselves, etc. I think a routine to get them is not a too bad idea.)

However, this is the routine which causes my current head-ache:

'Expects to find a structure of type T at the actual position of the 
'specified filestream oFS. Reads this structure into a byte array and copies
'it to the structure variable specified in oStruct.
Public Shared Sub ExtractStructure(Of T As Structure) _
    (oFS As FileStream, ByRef oStruct As T)

    Dim oGCHandle As GCHandle
    Dim oStructAddr As IntPtr
    Dim iStructLen As Integer
    Dim abStreamData As Byte()

    'Obtain a handle to the structure, pinning it so that the garbage
    'collector does not move the object. This allows the address of the 
    'pinned structure to be taken. Requires the use of Free when done.
    oGCHandle = GCHandle.Alloc(oStruct, GCHandleType.Pinned)

    Try
        'Retrieve the address of the pinned structure, and its size in bytes.
        oStructAddr = oGCHandle.AddrOfPinnedObject
        iStructLen = SizeOf(oStruct)

        'From the file's current position, obtain the number of bytes 
        'required to fill the structure.
        ReDim abStreamData(0 To iStructLen - 1)
        oFS.Read(abStreamData, 0, iStructLen)

        'Now both the source data is available (abStreamData) as well as an 
        'address to which to copy it (oStructAddr). Marshal.Copy will do the
        'copying.
        Marshal.Copy(abStreamData, 0, oStructAddr, iStructLen)
    Finally
        'Release the obtained GCHandle.
        oGCHandle.Free()
    End Try
End Sub

The instruction

oFS.Read(abStreamData, 0, iStructLen)

does read the correct number of bytes with the correct values, as per immediate window:

?abstreamdata
{Length=8}
    (0): 1
    (1): 0
    (2): 2
    (3): 0
    (4): 3
    (5): 0
    (6): 0
    (7): 0

I can not use Marshal.StructureToPtr here, because, well, a bytes array is not a structure. However, the Marshal class has also a Copy method.

Only that I obviously miss a point here, because

Marshal.Copy(abStreamData, 0, oStructAddr, iStructLen)

does not do the intended copy (but it also does not throw an exception):

?ostruct
    Count: 0
    MajorVersion: 0
    MinorVersion: 0

What do I fail to understand, why this copy is not working?

The MS documentation does not tell too much for Marshal.Copy: https://msdn.microsoft.com/en-us/library/ms146625(v=vs.110).aspx:

source - Type: System.Byte() The one-dimensional array to copy from.

startIndex - Type: System.Int32 The zero-based index in the source array where copying should start.

destination - Type: System.IntPtr The memory pointer to copy to.

length - Type: System.Int32 The number of array elements to copy.

Looks as if I have set up right everything, so is it possible that Marshal.Copy does not copy to the structure, but to some other place?

oStructAddr surely does look like an address (&H02EB7B64), but where is that?

Edit

Between the instantiation of the parameter receiving the result and the call to the routine

    grHeaderStruct = New HeaderStruct
    ExtractStructure(Of HeaderStruct)(oFS, grHeaderStruct)

I filled the instantiated record with some test data to see whether it is actually passed correctly:

#Region "Test"
    With grMultifileDesc
        .MajorVersion = 7
        .MinorVersion = 8
        .Count = 9
    End With
#End Region

In the procedure, I check the value of the record before and after Marshal.Copy in the immediate window. Both times I obtain:

?ostruct
{MyProject.MyClass.HeaderStruct}
    Count: 9
    MajorVersion: 7
    MinorVersion: 8

(Re-arrangement of the record's fields is the same in the caller as in the callee, which of course is an issue in itself, as the data is read from a file and would be wrongly copied into the structure. However, this is not the topic of the question.)

Conclusion: Data obtained, but no Marshal.Copy made already in the callee. So it is not just a "returns wrong data" problem.

Edit 2

It turns out,that Marshal.Copy does not copy data, as stated in the documentation, but a pointer to the data.

Marshal.ReadByte(oStructAddr) does return the byte array's first byte, Marshal.ReadByte(oStructAddr + 1) its second byte, etc.

But how do I return this data in the Out argument?

回答1:

I can not say for certain why the updated bytes copied to the pinned address are not reflected in the original structure, but I suspect that the interop-marshaller makes a copy. see: Copying and Pinning.

I do know that if you pin a byte array, that changes made by using Marshal.Copy are propagated back to the managed array. With that said, here is an example that you should be able to use for your needs.

Dim lenBuffer As Int32 = Marshal.SizeOf(Of HeaderStruct)
Dim buffer As Byte() = New Byte(0 To lenBuffer - 1) {}
Dim gchBuffer As GCHandle = GCHandle.Alloc(buffer, GCHandleType.Pinned)
Dim hs As New HeaderStruct With {.MajorVersion = 4, .MinorVersion = 2, .Count = 5}

' copy to pinned byte array 
Marshal.StructureToPtr(hs, gchBuffer.AddrOfPinnedObject, True)
' buffer = {4, 0, 2, 0, 5, 0, 0, 0} ' array can be written to file

' change the data - simulates array read from file
buffer(0) = 1 ' MajorVersion = 1
buffer(2) = 3 ' MinorVersion = 3
buffer(4) = 2 '.Count = 2

' read from pinned byte array
Dim hs2 As HeaderStruct = Marshal.PtrToStructure(Of HeaderStruct)(gchBuffer.AddrOfPinnedObject)
gchBuffer.Free()

Edit: Based on a comment, it seems that the OP believes that the technique shown above can not be implemented as generic methods for reading/writing to a stream. So here is a copy/paste example.

Sub Example()
    Using ms As New IO.MemoryStream
        Dim hs As New HeaderStruct With {.MajorVersion = 4, .MinorVersion = 2, .Count = 5}
        Debug.Print($"Saved structure:  {hs}")
        WriteStruct(ms, hs)  'write structure to stream

        ms.Position = 0 ' reposition stream to start 
        Dim hs2 As New HeaderStruct ' target for reading from stream
        ReadStruct(ms, hs2)
        Debug.Print($"Retrieved structure: {hs2}")
    End Using
End Sub

Sub WriteStruct(Of T As {Structure})(strm As IO.Stream, ByRef struct As T)
    Dim lenBuffer As Int32 = Marshal.SizeOf(Of T)
    Dim buffer As Byte() = New Byte(0 To lenBuffer - 1) {}
    Dim gchBuffer As GCHandle = GCHandle.Alloc(buffer, GCHandleType.Pinned)
    Marshal.StructureToPtr(struct, gchBuffer.AddrOfPinnedObject, True)
    strm.Write(buffer, 0, lenBuffer)
    gchBuffer.Free()
End Sub

Sub ReadStruct(Of T As {Structure})(strm As IO.Stream, ByRef struct As T)
    Dim lenBuffer As Int32 = Marshal.SizeOf(Of T)
    Dim buffer As Byte() = New Byte(0 To lenBuffer - 1) {}
    strm.Read(buffer, 0, buffer.Length)
    Dim gchBuffer As GCHandle = GCHandle.Alloc(buffer, GCHandleType.Pinned)
    struct = Marshal.PtrToStructure(Of T)(gchBuffer.AddrOfPinnedObject)
    gchBuffer.Free()
End Sub

Public Structure HeaderStruct
    Public MajorVersion As Short
    Public MinorVersion As Short
    Public Count As Integer
    Public Overrides Function ToString() As String
        Return $"MajorVersion = {MajorVersion}, MinorVersion = {MinorVersion}, Count = {Count}"
    End Function
End Structure

Output of Example:

Saved structure: MajorVersion = 4, MinorVersion = 2, Count = 5

Retrieved structure: MajorVersion = 4, MinorVersion = 2, Count = 5



回答2:

The ExtractStructure method needs to be rewritten like this:

'Expects to find a structure of type T at the actual position of the 
'specified filestream oFS. Reads this structure into a byte array and copies
'it to the structure variable specified in oStruct.
Public Shared Sub ExtractStructure(Of T As Structure) _
    (oFS As FileStream, ByRef oStruct As T)

    Dim iStructLen As Integer
    Dim abStreamData As Byte()
    Dim hStreamData As GCHandle
    Dim iStreamData As IntPtr

    'From the file's current position, read the number of bytes required to
    'fill the structure, into the byte array abStreamData.
    iStructLen = Marshal.SizeOf(oStruct)
    ReDim abStreamData(0 To iStructLen - 1)
    oFS.Read(abStreamData, 0, iStructLen)

    'Obtain a handle to the byte array, pinning it so that the garbage
    'collector does not move the object. This allows the address of the 
    'pinned structure to be taken. Requires the use of Free when done.
    hStreamData = GCHandle.Alloc(abStreamData, GCHandleType.Pinned)
    Try
        'Obtain the byte array's address.
        iStreamData = hStreamData.AddrOfPinnedObject()

        'Copy the byte array into the record.
        oStruct = Marshal.PtrToStructure(Of T)(iStreamData)
    Finally
        hStreamData.Free()
    End Try
End Sub

Works.