Process hangs pinvoking AmsiScanBuffer from manage

Question

I am attempting to use the AmsiScanBuffer function of the Windows Anti-Malware Service Interface from managed code, specifically C#. When attempting to call the method the program hangs on the call anytime a non-zero buffer length is provided. If a buffer length of 0 zero is provided then the method returns immediately with the HResult E_INVALIDARG. The other methods exposed by AMSI work as expected so I expect the I believe my dllimport for this function to be close but probably not completely correct. Besides the array copy approach represented here I have tried pinning the array and the behavior is identical.

C prototype

HRESULT WINAPI AmsiScanBuffer(
  _In_     HAMSICONTEXT amsiContext,
  _In_     PVOID        buffer,
  _In_     ULONG        length,
  _In_     LPCWSTR      contentName,
  _In_opt_ HAMSISESSION session,
  _Out_    AMSI_RESULT  *result
);

Managed Code

[DllImport("Amsi.dll", EntryPoint = "AmsiScanBuffer", CallingConvention = CallingConvention.StdCall)]
public static extern int ScanBuffer(IntPtr amsiContext, IntPtr ptr, ulong length, string contentName, IntPtr session, out int result);

var virus = "X5O!P%@AP[4\\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*";
var bytes = Encoding.UTF8.GetBytes(virus);
int sizet = Marshal.SizeOf(typeof(byte)) * bytes.Length;
var ptr = Marshal.AllocHGlobal(sizet);

try
{
    Marshal.Copy(bytes, 0, ptr, bytes.Length);
    int hr = Amsi.ScanBuffer(context, ptr, (ulong)sizet, "Unknown Data", session, out result);
}
finally
{
    Marshal.FreeHGlobal(ptr);
}

Answer 1

The main problem is the length parameter to AmsiScanBuffer. A ULONG in C/C++ on Windows is 32 bits, while a ulong in C# is 64 bits. So the parameter needs to be declared as a uint. I would have expected you'd get an "unbalanced stack" error when running under the debugger even if you passed a buffer length of zero. You can also declare buffer as a byte[] and then just pass in the bytes directly.

For further simplification, you can omit the CallingConvention since StdCall is the default. I also changed it to use the exact function name so it isn't necessary to specify it in the DllImport. In general, when I'm working with C libraries directly from C# I like to keep the original function names, e.g. AmsiScanBuffer instead of changing it to Amsi.ScanBuffer. This makes it easier to look up docs when somebody is working on the code, although this is of course a matter of taste.

Here's a working version as a console application.

using System;
using System.Runtime.InteropServices;
using System.Text;

namespace AmsiTest {

    class Program {
        static void Main( string[] args ) {

            var virus = Encoding.UTF8.GetBytes(
                "X5O!P%@AP[4\\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*"
            );

            IntPtr context;
            var hrInit = AmsiInitialize( "AmsiTest", out context );
            if( hrInit != 0 ) {
                Console.WriteLine( $"AmsiInitialize failed, HRESULT {hrInit:X8}" );
                return;
            }

            AMSI_RESULT result;
            var hrScan = AmsiScanBuffer(
                context, virus, (uint)virus.Length,
                "EICAR Test File", IntPtr.Zero, out result
            );

            AmsiUninitialize( context );

            if( hrScan != 0 ) {
                Console.WriteLine( $"AmsiScanBuffer failed, HRESULT {hrScan:X8}" );
            } else if( result == AMSI_RESULT.AMSI_RESULT_DETECTED ) {
                Console.WriteLine( "Detected EICAR test" );
            } else {
                Console.WriteLine( $"Failed to detect EICAR test, result {result:X8}" );
            }

        }

        public enum AMSI_RESULT { 
            AMSI_RESULT_CLEAN = 0,
            AMSI_RESULT_NOT_DETECTED = 1,
            AMSI_RESULT_BLOCKED_BY_ADMIN_START = 16384,
            AMSI_RESULT_BLOCKED_BY_ADMIN_END = 20479,
            AMSI_RESULT_DETECTED = 32768
        }

        [DllImport( "Amsi.dll" )]
        public static extern uint AmsiInitialize(
            string appName,
            out IntPtr amsiContext
        );

        [DllImport( "Amsi.dll" )]
        public static extern uint AmsiScanBuffer(
            IntPtr amsiContext,
            byte[] buffer,
            uint length,
            string contentName,
            IntPtr session,
            out AMSI_RESULT result
        );

        [DllImport( "Amsi.dll" )]
        public static extern void AmsiUninitialize(
            IntPtr amsiContext
        );
    }
}