Windows bindings

[Jonathan Marler]

On Saturday, 13 February 2021 at 00:32:30 UTC, Rumbu wrote:
> I recently started a new project for Windows bindings [1] based 
> on metadata generated by Windows Metadata Project [2].
>

Very cool I've been working with this project as well.  I decided 
to create a project that converts the winmd file to JSON (see 
https://github.com/marlersoft/win32json).  This makes it easy to 
search through the data with grep and allows it to be more easily 
consumed by other tools.

> I succesfully managed to implement a winmd reader in D (it 
> works also with any DLL file containing CLI metadata) and 
> generated the first bindings [3]

Great job on that, this looks like it was a good chunk of work.  
Out of curiosity, did you write this from the spec and/or did you 
reference other implementations?

>
> Now, during the process of generation, a lot of questions 
> popped out and I'm asking for community help to sort them out.
>
> 1) On In/Out/Optional attributes
>
> Most of the pointer function parameters are decorated with any 
> combination of these 3 attributes and I don't have any idea how 
> to represent them or find them any use.
>
> For the "in" attribute the direct equivalent will be 'const' 
> but usually this kind of parameters are already decorated with 
> another attribute called 'IsConst'.


I've found that [In] doesn't necessarily mean "const".  There 
seem to be many in parameters that are marked "[In]" but are not 
const. That being said, not all parameters that are suppsed to be 
const are marked correctly, but I believe this is because 
win32metadata is still pretty new and working out issues like 
that.  For example, the buffer parameter for 
WriteFile/WriteFileEx was not marked as "Const", so I submited a 
PR that fixed it here: 
https://github.com/microsoft/win32metadata/pull/212

>
> For the Out attribute I think that an option will be to 
> derefrence the pointer and put a 'out' parameter qualifier 
> instead. Example:
> - void foo(int* val) becomes void foo(out int val)
> The problem appears when I have prototypes like foo(char*) 
> where the poimnter is in 99% of cases a pointer to a null 
> terminated string
>
> For the Optional attribute I have no idea except that put null 
> as default:
> - void foo(int* val) becomes void foo(int* val = null)
> But this will work only for terminal parameters, if the 
> optional parameter is in the middle of the list, there is no 
> way to do that.

For pointer types I think this really means "nullable".  One way 
you could do it is if a pointer parameter is not marked as 
"Optional", then you could make it a "ref" type rather than a 
pointer, but, that might have other unintended consequences.

>
>
> 2) On ComOutPtr attribute
> I didn't find any use of this attribute, it usually decorates 
> parameters returning COM interfaces. Example, the ppv parameter 
> below is decorated with this attribute:
>
> HRESULT foo(IUnknown* ppv);

I believe this is marking parameters that are returning COM 
objects.  Maybe the reason for this is that some functions may 
return COM objects, but it may return multiple kinds of COM 
objects, so it can't declare that it's a COM object through it's 
type?

>
>
> 3) On enums
>
> None of the enums are anonymous, every old Windows constant now 
> is part of an enum which results in very long names difficult 
> to write. Example:
>
> enum FILE_NOTIFY_CHANGE
> {
>     FILE_NOTIFY_CHANGE_FILE_NAME = 1,
>     FILE_NOTIFY_CHANGE_DIR_NAME = 2,
>     FILE_NOTIFY_CHANGE_ATTRIBUTES = 4,
>     FILE_NOTIFY_CHANGE_SIZE = 8,
>     FILE_NOTIFY_CHANGE_LAST_WRITE = 16,
>     FILE_NOTIFY_CHANGE_LAST_ACCESS = 32,
>     FILE_NOTIFY_CHANGE_CREATION = 64,
>     FILE_NOTIFY_CHANGE_SECURITY = 256,
> }
>
> A first option will be to drop the enum name and come back to 
> the old windows constants.
>
> A second option, which seems more interesting, is to drop the 
> member prefix when it coresponds to the enum name.
>
> enum FILE_NOTIFY_CHANGE
> {
>     FILE_NAME = 1,
>     DIR_NAME = 2,
>     ATTRIBUTES = 4,
>     ...
> }
>
> Unfortunately this option is not always available because some 
> enums are named with ENUM or other tags at the end or the enum 
> name is completely different compared with the members within.

For the bindings I've generated I went with the second option, 
but I also declared the full constants as well.  So your example 
becomes:

enum FILE_NOTIFY_CHANGE
{
     FILE_NAME = 1,
     DIR_NAME = 2,
     ATTRIBUTES = 4,
     ...
}
alias FILE_NOTIFY_CHANGE_FILE_NAME = FILE_NOTIFY_CHANGE.FILE_NAME;
alias FILE_NOTIFY_CHANGE_DIR_NAME = FILE_NOTIFY_CHANGE.DIR_NAME;
alias FILE_NOTIFY_CHANGE_ATTRIBUTES = 
FILE_NOTIFY_CHANGE.ATTRIBUTES;
...

Not sure what to do yet about the weird enums whose values don't 
match the name of the enum, maybe they need to be addressed on a 
case by case basis?

>
> 4) On RAIFree attribute
>
> Most of the specific handles are decorated with this attribute 
> stating the function meant to free the handle. Example:
>
> [RAIIFree(CloseDC)]
> struct HDC ...
>
> Sincerely, it's interesting concept (to know what function to 
> use in a potential destructor), but I have no idea how can I 
> take advantage on this.
>

One thing you could do is define a common method that calls it:

struct HDC
{
     auto free()
     {
         return CloseDC(this.value);
     }
}

struct HANDLE
{
     auto free()
     {
         return CloseHandle(this.value);
     }
}

It's not really a big feature, but it means you can free any type 
with this attribute the same way.  Could make them easier to use 
in template code.


>
> 5) On DllImport attribute
>
> Each function is decorated with a DllImport attribute stating 
> the dll file where it can be found. If a dynamic binding is 
> intended later, this can be useful. All I've done now was to 
> decorate also in D each function with the same attribute. 
> Example:
>
>
> @DllImport("WININET.dll")
> BOOL InternetTimeToSystemTimeA(const(char)* lpszTime, 
> SYSTEMTIME* pst, uint dwReserved);
>
> Maybe with some traits magic, we can obtain a pointer to the 
> function at runtime.

There is an interesting feature you might be able to implement 
something with. A while back I worked on my own set of Windows 
bindings and I developed a pattern where I categorized things by 
whether or not they required linking to a library.  Here's the 
modules for kernel32: 
https://github.com/dragon-lang/mar/tree/master/src/mar/windows/kernel32

The "link.d" module starts with `pragma(lib, "kernel32.lib");`.  
I put all the function definitions in link.d so when it gets 
imported, kernel32.lib automatically gets added to the library 
link list.  Then I put all the type/const definitions in 
"nolink.d" because they can be used without having to link to 
kernel32.lib. This means the libraries you use and don't use get 
added and removed purely based on what your program is calling.

That being said, win32metadata is not split by library but 
instead by category, so I'm not sure how this would work since 
you're generating modules based on the category as well.  Maybe 
you could sort all the functions into their own empty templates 
based on which library they are in and put the pragmas in there; 
something like this?

template kernel32()
{
     pragma(lib, "kernel32.lib");
     void WriteFile(...);
     void WriteFileEx(...);
}
alias WriteFile = kernel32!().WriteFile;
alias WriteFileEx = kernel32!().WriteFileEx;

template user32()
{
     pragma(lib, "user32.lib");
     void SomethingElse(...);
     void AnotherThing(...);
     void AlsoThis(...);
}
alias SomethingElse = user32!().SomethingElse;
alias AnotherThing = user32!().AnotherThing;
alias AlsoThis = user32!().AlsoThis;

This might be overkill, but food for thought.

>
>
> 6) On GUID attribute
>
> There are no IID or CLSID constants declared in the metadata. 
> Instead,
> - each interface is decorated with a GUID attribute 
> corresponding to it
> - for COM classes, an empty struct is declared and decorated 
> with that attribute.
>
> For interfaces I created the corresponding constant named 
> IID_InterfaceName and for classes CLSID_ClassName.
> I also left attached as an attribute the Guid, thinking about 
> the fact that some trait magic can extract an interface 
> constant with a syntax like "uuidof!InterfaceName" (i've seen 
> this in some projects).
>
> So finaly an interface looks translated in D code like this:
>
> const GUID IID_ID3D12RootSignature = {0xC54A6B66, 0x72DF, 
> 0x4EE8, [0x8B, 0xE5, 0xA9, 0x46, 0xA1, 0x42, 0x92, 0x14]};
> @GUID(0xC54A6B66, 0x72DF, 0x4EE8, [0x8B, 0xE5, 0xA9, 0x46, 
> 0xA1, 0x42, 0x92, 0x14]);
> interface ID3D12RootSignature : ID3D12DeviceChild
> {
> ....
> }


IID_* and CLSID_* are actually supposed to be pointers to GUID's 
rather than GUIDS themselves.  So you might want to do something 
like:

immutable GUID IID_ID3D12RootSignature_Value = {0xC54A6B66, 
0x72DF, 0x4EE8, [0x8B, 0xE5, 0xA9, 0x46, 0xA1, 0x42, 0x92, 0x14]};
immutable GUID* IID_ID3D12RootSignature = 
&IID_ID3D12RootSignature_Value;

Or maybe create some sort of template that create storage for a 
guid and returns a pointer to it.


>
> 7) On strongly typed structs
>
> In the old Windows days, handles were simply typedefs to 
> pointers. Now another idea emerged, For example, a HFONT handle 
> which in the past was simply a void*, now is described like 
> this:
>
> struct HFONT
> {
>    ptrdiff_t value;
> }
>
> This will oblige to HFONT related functions to use only  this 
> struct as parameter and thi struct will always have the size of 
> a pointer.
>
> Sincerely I don't have any idea how to translate this to D, 
> currently I simply put an alias HDC = ptrdiff_t;

Yeah this one is interesting.  Because some functions like 
CreateWindow and types like WNDCLASS take something like HMODULE 
but are typically assigned a value of type HINSTANCE.  So in some 
cases, each type having their own struct wrapper could be 
helpful, and in other cases it could just mean alot of 
boilerplate to cast between types.  I'll have to play with this 
more myself to see what's the way to go.