WindowsAPI - Problem with DECLARE_HANDLE definition

[Vladimir Panteleev]

On Sunday, 8 September 2013 at 23:52:46 UTC, Stewart Gordon wrote:
> 1. Define a hierarchy of dummy classes for the handle types.  
> No actual objects will exist of these types, but since classes 
> are reference types they can be set to null.
>
> But there's a nasty bug lurking in this: if somebody tries to 
> compare handles using ==, it will dereference the pointer, and 
> look in vain for the vtable and the opEquals method defined 
> therewithin ... cue major chaos.

This is a showstopper. It breaks existing code in subtle ways.

> 2. Do 1, but use pointers to these classes as the handle types.
>
>     class HANDLE_ {}
>     alias const(HANDLE_)* HANDLE;
>     class HWND_ : HANDLE_ {}
>     alias const(HWND_)* HWND;
>
> This would avoid the dereferencing behaviour.  It's to be hoped 
> that all Windows programmers know that, although handles are 
> declared as pointer types, they cannot meaningfully be 
> dereferenced.  But what would the GC do, especially given that 
> there are two levels of indirection neither of which points to 
> an appropriate memory location?

This is fine as far as the GC goes. Error messages are not great, 
though: "cannot implicitly convert expression (h) of type 
const(HANDLE_)* to const(HWND_)*". Not sure about object code - 
one possibility would be to fix DMD to support class declarations 
(e.g. "class HWND_ : HANDLE_;" with no body).

> 3. Keep the current implementation, but implement an enum 
> member NULL in each handle type, like this:
>
>     struct HWND {
>         HANDLE h;
>         alias h this;
>         enum HWND NULL = cast(HWND) 0;
>     }
>
> Programmers still can't use null, but writing HWND.NULL might 
> be acceptable as the next best thing.

This is not much better than writing HWND.init (which is how I've 
been working around the issues).

> 4. Abandon this hierarchy idea and just define DECLARE_HANDLE 
> the same way as the MinGW C headers do.

Supporting the functionality of the STRICT define would be a good 
start; I don't think that involves any hierarchies. I think an 
important goal is to avoid breaking most correct code and 
maximize compatibility with C code, so using a special literal 
for null is out.

By the way, another problem with the current implementation is 
casts. At the moment, they can get quite verbose, and confusing 
to implement (one needs to look up the hierarchy of the handle 
types).

For example, calling LocalFree with a LPCWSTR as when using 
FormatMessage now looks like this:

	LocalFree(HLOCAL(HANDLE(lpMsgBuf)));

Having the handle types be some kind of pointer would allow using 
a more conventional casting syntax.