Passing an unmanaged pointer between two AppDomain

Question

.NET 4.5, 64bit Win8

I have two MSMQ running under WCF and hosted under Windows Activation Service.

1. Execute MSMQ: Responsible for calling an unmanaged function and obtaining a pointer IntPtr to 4GB array in addition to other limited size variables. This array is not that important and shouldn't be blocking the queue until it is passed to the DB and having this queue free is a priority. Passing the pointer to the Save MSMQ (Hypothesis).
2. Save MSMQ: Should save the array to the DB across the network then deallocate it from unmanaged memory.

The problem here would be:

1. Making sure the array is deallocated on poison messages and other queue errors. This case is the easy bit.
2. Passing an IntPtr cross app domains which might not be possible, as I recall, from the old good days, that pointer values are relative (and the word offsetting comes to mind, or maybe I am mixing processes and AppDomains) to app domains and for security reasons .NET might not allow this? Is this possible or am I dreaming, also is this a cross domain issue or does my problem have a different title?

N.B. I have used NetPipes communication before, but this has a queue in the middle, so NetPipes would not be a good solution.