Why can Delphi DLLs use WideString without using S

Question

David's answer to another question shows a Delphi DLL function returning a WideString. I never thought that was possible without the use of ShareMem.

My test DLL:

function SomeFunction1: Widestring; stdcall;
begin
  Result := 'Hello';
end;

function SomeFunction2(var OutVar: Widestring): BOOL; stdcall;
begin
  OutVar := 'Hello';
  Result := True;
end;

My caller program:

function SomeFunction1: WideString; stdcall; external 'Test.dll';
function SomeFunction2(var OutVar: Widestring): BOOL; stdcall; external 'Test.dll';

procedure TForm1.Button1Click(Sender: TObject);
var
  W: WideString;
begin
  ShowMessage(SomeFunction1);
  SomeFunction2(W);
  ShowMessage(W);
end;

It works, and I don't understand how. The convention I know of is the one used by the Windows API, for example Windows GetClassNameW:

function GetClassNameW(hWnd: HWND; lpClassName: PWideChar; nMaxCount: Integer): Integer; stdcall;

Meaning the caller provides the buffer, and the maximum length. The Windows DLL writes to that buffer with the length limitation. The caller is allocates and deallocates the memory.

Another option is that the DLL allocate the memory for example by using LocalAlloc, and the Caller deallocates the memory by calling LocalFree.

How does the memory allocation and deallocation work with my DLL example? Does the "magic" happen because the result is WideString(BSTR)? And why aren't Windows APIs declared with such convenient convention? (Are there any known Win32 APIs that uses such convention?)

---

EDIT:

I Tested the DLL with C#.
Calling SomeFunction1 causes an AV (Attempted to read or write protected memory).
SomeFunction2 works fine.

[DllImport(@"Test.dll")]
[return: MarshalAs(UnmanagedType.BStr)]
static extern string SomeFunction1();

[DllImport(@"Test.dll")]
[return: MarshalAs(UnmanagedType.Bool)]
static extern bool SomeFunction2([MarshalAs(UnmanagedType.BStr)] out string res);

...

string s;
SomeFunction2(out s);
MessageBox.Show(s); // works ok
MessageBox.Show(SomeFunction1()); // fails with AV!

Here is a followup.

Answer 1

A WideString is the same as a BSTR, it's just the Delphi name for it. The memory allocation is handled by the shared COM allocator, CoTaskMemAlloc. Because all parties use the same allocator you can safely allocate in one module and deallocate in another.

So, the reason you don't need to use Sharemem is that the Delphi heap is not being used. Instead the COM heap is used. And that is shared between all modules in a process.

If you look at the Delphi implementation of WideString you will see calls to the following APIs: SysAllocStringLen, SysFreeString and SysReAllocStringLen. These are the system provided BSTR API functions.

Many of the Windows APIs you refer to pre-date the invention of COM. What's more, there are performance benefits to using a fixed length buffer, allocated by the caller. Namely that it can be allocated on the stack rather than a heap. I also can imagine that the Windows designers don't want to force every process to have to link to OleAut32.dll and pay the price of maintaining the COM heap. Remember that when most of the Windows API was designed, the performance characteristics of the typical hardware was very different from now.

Another possible reason for not using BSTR more widely is that the Windows API is targeted at C. And managing the lifetime of BSTR from C is very much more tricky than from higher level languages like C++, C#, Delphi etc.

There is an extra complication however. The Delphi ABI for WideString return values is not compatible with Microsoft tools. You should not use WideString as a return type, instead return it via an out parameter. For more details see Why can a WideString not be used as a function return value for interop?