Higher abstraction level for calling C functions

[Timon Gehr]

On 01/29/2012 09:48 PM, Denis Shelomovskij wrote:
> D has complete (IMHO) compiler support for calling C functions (using
> extern(C)). But there is a lack of library support.
>
> Readers from this NG probably know this, but... Microsoft .NET Framework
> has such support. Look briefly at source example in:
> http://msdn.microsoft.com/en-gb/library/s97shtze.aspx
>
> And at System.IO.Directory.GetCurrentDirectory source:
> http://typedescriptor.net/name/members/5540086E017CD13896E80A0CAEA6E517-System.IO.Directory.GetCurrentDirectory%28%29
>
> where Win32Native.GetCurrentDirectory is defined as:
> ---
> [DllImport("kernel32.dll", CharSet=CharSet.Auto, SetLastError=true)]
> internal static extern int GetCurrentDirectory(int nBufferLength,
> StringBuilder lpBuffer);
> ---
>
> Looks good? But it's not. It tries to provide both low level support
> (it's needed to just call C function, like D extern(C), but do
> everything at runtime (yes, in .NET Framework even Regex can be compiled
> at runtime from a string)) and high level support (it converts UTF-16 to
> ANSI in the given MSDN example). As a result it even doesn't allow you
> to specify a (scalar) parameter as a length of another (array) parameter
> or throw on native function failure.
>
>
>
> What can be done in that direction in D? The whole my point is expressed
> in my small cwrap library:
> http://deoma-cmd.ru/d/docs/src/cwrap.html
>
> "Examples" is the most informative section of the documentation. Other
> examples can be found in (original functions, all in one file):
> https://bitbucket.org/denis_sh/cwrap/src/tip/examples/c.d
> and generated wrappers:
> https://bitbucket.org/denis_sh/cwrap/src/tip/out
>
> Library state: almost every internal function is unittested, generates
> compilable code which looks right but not unittested yet.
> NOTE: You need head dmd to CTFE it or you can just change `enum data` to
> `auto data` in examples/c.d, this will also reduce compilation time more
> than x10.
>
> Once original function is properly described in IDL, such library gives
> the fallowing advantages:
> * [memory-corruption-safe] User works with arrays, not separate pointers
> and lengths.
> * [memory-leak-safe] Original function allocated memory will be freed on
> any failure (NOTE: see "Original function restrictions" in documentation).
> * [fail-safe] An exception will be thrown if original function fails.
> * [usability] Much easier to work with arrays/preferred string types (it
> converts UTF-8/16/32 back and forth, see docs).
> * [speed] No unnecessary GC (or other) allocations/copying/reading.
> * [garbage-free] Look at, e.g. std.file - it leaves lots of garbage
> because of calling native platform functions. Generated wrappers don't
> even allocate GC memory (except for output parameters).
>
> Looks like this functionality is usable for auto-generated bindings, at
> least a useful GtkD still leaks memory almost everywhere because it
> never frees (OK, not sure, I just haven't seen) allocated memory marked
> by "free with g_free".
>
> So should such stuff be ever included in Phobos? IMHO yes because it's a
> general stuff.

+1.

> But if not, can generated wrappers only be included and
> used to improve current situation "a system language with a library
> which leaves garbage"?
>
> By the way, std.file.getcwd and std.windows.charset.toMBSz are examples
> of hand-written wrappers (the first is mine and strictly speaking forced
> me to do this lib).
>
> P.S.
> Not sure if this library usable at all because I'm as sure in its
> usability as in tabs indention superiority over spaces indention. )
> So if it's not (usable), tell me without scruple.
>
>
> Thanks for reading!

Looks like very cool stuff.

In this example:

// from cstring.h
__(out) int memcmp(
	__(in[num] ) void*  ptr1,
	__(in[+num]) void*  ptr2,
	__(in      ) size_t num
)
__( success: true );

I suppose the unary + before the second 'num' is required to 
disambiguate from which array length 'num' will be deduced? Shouldn't 
this restriction be lifted? (Obviously, if the lengths have to match, 
both are fine.)

In this example:

// from WinBase.h
__(out) DWORD GetCurrentDirectoryW(
	__(in) DWORD nBufferLength,
	__(out""[nBufferLength]->""[return]) LPWSTR lpBuffer
)
__( repeat: return > nBufferLength )
__( success: return );
// NOTE: current directory can't have zero length

Maybe the generated bindings should rather look like

char[] winapi_GetCurrentDirectoryW();