C++ interface vs D and com

[Adam Sansier via Digitalmars-d-learn]

On Wednesday, 13 July 2016 at 08:34:55 UTC, John wrote:
> On Wednesday, 13 July 2016 at 07:31:57 UTC, Adam Sansier wrote:
>> void** ptr = null;		
>> auto res = CoCreateInstance(&CLS_ID, cast(IUnknown)null, 
>> CLSCTX_INPROC_SERVER, &CLS_ID, cast(void**)&ptr);
>
> How are you casting your "ptr" variable (which BTW should be 
> just void* or usually IUnknown) to your interface? A common 
> mistake if you're used to C++ is using cast(ISomeInterface*)ptr 
> - which would cause access violations - instead of 
> cast(ISomeInterface)ptr.


No, I figured that stuff out about the first 5 mins I started 
this 3 day journey.

This is the problem, and I believe the solution:

The COM interface I'm using does not use this in the code and 
either is "static" no this passed or simply ignores it. I know in 
some cases when I would try certain things my arguments to them 
would not be correct(pass a char* to get data and get crap back 
or crash randomly).

IUnknown requires one to pass the interface as a this.

The mismatch between the two created most of the havoc as I 
wrongfully assumed they were all the same and, as typical with 
computers, the behavior was not consistent and sometimes would 
work in some ways and other times wouldn't.

Regardless, to solve the problem, I believe(No crashes, don't 
know if the references are truly handled properly, I have to do 
this:


extern (C++) class cASIO
{
		extern (C++)
		{
			int QueryInterface(void*, const(GUID)* riid, void** pvObject)
			{
				return 0;
			}
			uint AddRef(void*)
			{
				return 0;
			}
			uint Release(void*)
			{
				return 0;
			}
		}
		extern(C++)
		{
			void func1(char *name) { }
			void func2(int) { }
                         ...
                 }
}

extern(C++) doesn't pass this or passes it in ECX(Not sure which 
yet, I think it doesn't pass this from what I recall about the 
disassembly).

I then simply manually pass the interface ptr created by 
CoCreateInstance to Release.

Of course, if I could cast the interface ptr directly to a type 
with the interface and it all work, then D should handle that 
stuff behind the scenes. I tried that initially and it might have 
worked or half worked or something.  I will go back and try again 
knowing what I know now and try to keep everything correct.

So, for those that run into these COM problems in the future:

1. Make sure the linkage is 100% correct per function. IUnknown 
uses extern(Windows) but must be passed the correct this or it 
will not work well. This can be emulated by marking them 
extern(C++) and passing the expected this as first parameter.

The actual interface may not have the same linkage as IUnknown. 
If the 0 parameter functions are working but non-zero parameter 
functions are not behaving correctly then is a linkage issue. 
They should be marked extern(C++) as this gives more control over 
what is passed. Passing the correct this as the first parameter 
will either work or not, if not, either the function expects it 
in ECX or it is some type of static like function. This can be 
difficult to know if the function itself doesn't even use this 
internally.

One can debug these things and see the vtable in memory and all 
the function addresses and be sure the addresses are linking up, 
but one can't see the parameter passing as easily. If your sure 
the function addresses are correct and your not calling the wrong 
addresses, then the problem is a linkage issue.

2. The interface ptr returned by COM should be able to be 
mappable to a *COM* interface pointer in D. A normal interface 
differs by a COM interface in D the vtable is offset by a ptr. In 
this case, your calls may work or not depending on the design but 
something will eventually not work causing grief. We have no idea 
how to mark an interface as COM in D... someone once said through 
the grapevine that it does this if it inherits IUnknown. I 
believe that marking it extern(C++) works.

3. There are a lot of pitfalls here because of the way the 
different modifiers work. Some combinations may work when they 
really don't... Trial and error is a real pain. Making sure the 
vtable is correctly being used and then knowing the linkage of 
the interface functions should get one at least 50% there.  This 
assumes different functions in the difference don't use different 
linkage... which may create even more havoc. There's a lot of 
misinformation on the net. Just because someone got some code to 
work in their specific case doesn't mean it is the correct way to 
do it for your case.