C++ interface vs D and com

[Adam Sansier via Digitalmars-d-learn]

On Wednesday, 13 July 2016 at 06:44:36 UTC, Adam Sansier wrote:
> On Wednesday, 13 July 2016 at 03:38:03 UTC, Mike Parker wrote:
>> On Wednesday, 13 July 2016 at 02:49:54 UTC, Adam Sansier wrote:
>>> On Wednesday, 13 July 2016 at 02:34:14 UTC, Mike Parker wrote:
>>
>>>> What happens when you declare an interface that extends from 
>>>> IUnknown (and not extern(C++)), then cast the pointer 
>>>> returned from the COM API? It should just work without 
>>>> needing to muck around with the vtable.
>>>
>>> That was what I tried first, It didn't work. I don't know 
>>> what the problem though. I either get an access violation or 
>>> the functions don't do anything.
>>
>> Perhaps you forgot to call CoInitialize{Ex}?
>
> Nope...
>
>>
>>>
>>> I think it's more complex because without extern(C++) the 
>>> vtable is in a different place than expected(it's offset by 
>>> 1), so simple casting does not work.
>>>
>>> "A COM interface differs from a regular interface in that 
>>> there is no object.Interface entry in vtbl[0]; the entries 
>>> vtbl[0..$] are all the virtual function pointers, in the 
>>> order that they were declared. This matches the COM object 
>>> layout used by Windows.
>>>
>>> A C++ interface differs from a regular interface in that it 
>>> matches the layout of a C++ class using single inheritance on 
>>> the target machine. "
>>
>> You don't need extern(C++) for COM interfaces. There are 
>> several declared in the Windows bindings that each inherit 
>> from IUnknown and there's no extern(C++) in sight (they 
>> existed long before C++ support did). Here's a working example 
>> using one of them, IShellLinkW, declared in 
>> core.sys.windows.shlobj.
>>
>> ```
>> import core.sys.windows.windows,
>>        core.sys.windows.shlobj,
>>        core.sys.windows.com;
>>
>> pragma(lib, "Ole32");
>>
>> void main()
>> {
>>     IShellLinkW iface;
>>     auto shellLinkCLSID = CLSID_ShellLink;
>>     auto shellLinkIID = IID_IShellLinkW;
>>
>>     CoInitialize(null);
>>     scope(exit)CoUninitialize();
>>
>>     auto hr = CoCreateInstance(
>>         &shellLinkCLSID,
>>         null,
>>         CLSCTX_INPROC_SERVER,
>>         &shellLinkIID,
>>         cast(void**)&iface
>>     );
>>     if(SUCCEEDED(hr)) {
>>         import std.stdio : writeln;
>>         writeln("Got it!");
>>         iface.Release();
>>     }
>>     else throw new Exception("Failed to create IShellLink 
>> instance");
>> }
>> ```
>>
>> There's a minor annoyance here in that the IID constants are 
>> all declared in the Windows bindings as manifest constants, 
>> which is normally the smart thing to do with constants. 
>> However, they're intended to be used as lvalues with the COM 
>> API, so I had to save them off in local variables in order to 
>> take their addresses. You can do whatever you want with your 
>> own declarations, of course.
>
> You don't have to beleive me, but if I don't mark the methods 
> extern(C++), then only 0 arg methods work.
>
> In fact, Release does not work unless I mark it extern (C++).
>
> So, while you may think it should work one way, and maybe it 
> does for you in some case, it doesn't for me and has given me 
> quite an amount of grief.
>
> Regardless of what you think, I can prove that the code won't 
> work when it is marked extern(Windows) and works when it is 
> marked extern (C++)... so what you should be asking yourself is 
> why it is doing that rather than assuming I'm making it up or 
> doing something wrong.

Ok, this is the thing.

In C++ I can do

		auto p = *((size_t**)*ptr) + 4;
		typedef size_t(__stdcall *fp)(char*);
		auto f = (fp)*p;
		res = f(n);

to call the 4th function that accepts a char. I can do that for 
all the functions > 2.

If I call 0 through 2 I get errors, that is from the IUnknown 
interface.

It seems the interface I'm using is built up of static 
functions... functions that don't accept a this. Hence they work 
and extern(C++) works for them, which I guess doesn't pass this.

Because of the hacked vtble stuff, the class used is not the 
original vtbl which is required for IUnknown.

Hence, they crash because they either get no this or the wrong 
this.

e.g.,
                 // Release the interface
		auto p = *((size_t**)*ptr) + 2;
		typedef size_t(__stdcall *fp)(void*);
		auto f = (fp)*p;
		res = f(ptr);


I'm not entirely sure if all this is correct but the C++ code 
shows that I don't pass a "this" to the interface functions and 
it works(if it's passed in ECX and they simply don't use it or if 
they are "static" or whatever is going on)... not sure which. 
OTOH, I have to pass something to Release to get it not to crash.

So, any ideas?

void** ptr = null;		
auto res = CoCreateInstance(&CLS_ID, cast(IUnknown)null, 
CLSCTX_INPROC_SERVER, &CLS_ID, cast(void**)&ptr);

I could check to see if addref and release are working if I could 
get the number of references. Is this possible? A quick search 
doesn't bring up anything.


To sum up this confusion:

1. Are the working interface functions I'm calling "static"(no 
this)? Is that possible? extern(Windows) breaks the code and 
extern(C++) works. I can call them with a C++ function pointer 
without this and it works. Not sure if this is passed on the 
stack or ecx and none of the functions I've called use this so 
they don't crash.

2. IUnknown's methods are crashing. This seems to be because this 
is not passed or an invalid this is passed.  This is probably due 
to the way I am hacking the vtbl but it is the way it was done in 
the C++ code. I can fix this stuff up without a big problem 
though as long as I know what to pass for this and possibly check 
to make sure the reference count decreases.

Thanks..