OT: Re: DConf Hackathon Ideas

[Nicholas Wilson via Digitalmars-d]

On Monday, 1 May 2017 at 12:35:11 UTC, Petar Kirov [ZombineDev] 
wrote:
> On Saturday, 29 April 2017 at 00:54:59 UTC, Nicholas Wilson 
> wrote:
>> On Friday, 28 April 2017 at 13:31:33 UTC, Petar Kirov 
>> [ZombineDev] wrote:
>>> Other applications include:
>>> * compiling/transpiling D functions to targets
>>> like JS, SPIR-V,
>>
>> I got you covered ;)
>
> I know, and I'm looking forward to using your GPU support in 
> LDC :P
>
>> (LDC not CTFE though. It would be fiendishly complicated to do 
>> at CTFE as a fair amount of compiler magic is necessary.)
>
> The way I see it, metaprogramming is a whole spectrum. Yes, you
> need a full compiler if you want to compile a whole program to
> e.g. JS or WebAsm, but there are many areas where even the 
> smallest
> improvement would be highly beneficial.
>

Oh I agree. For text to text (or lambda to text) this is 
comprehensible and not too difficult (perhaps even simple with 
Dmitry Olshansky's Pry) but SPIRV is rather complicated, as 
evidenced by a 203 page spec *(that doesn't even include the 
OpenCL or Vulkan specific stuff). Having said that it shouldn't 
be too difficult to port the tablegen tables I've been working on 
for LLVM to D (or write a D backend for tablegen) and get most of 
the really boring work out of the way.
I won't stop you, but just letting you know the rabbit hole is 
quite deep :)

The good news is the the runtime stuff will be all transferable.
This approach for SPIRV won't automatically translate for NVPTX 
like it will for ldc though, which is one of the main draws for 
dcompute

*Other languages probably have longer specs, but this is just for 
the format.

> Take for example C#. It has zero CTFE capabilities (you can
> safely ignore constant folding, which works only with string 
> and number literals),
> yet it has pretty powerful reflection and code-generation 
> capabilities
> at run-time. Even though the performance difference between CT 
> and RT
> reflection is orders of magnitude, those reflection 
> capabilities are used
> pervasively throughout the whole ecosystem. The prime example 
> being LINQ -
> probably the most widely used feature of .NET. Given a chain of 
> operations
> (similar to D's ranges) like:
>
> dbContext.Persons
>   .Where(p => p.Birthdate < DateTime.Now.Date.AddYears(-18))
>   .Where(p => p.Birthdate > DateTime.Now.Date.AddYears(-28))
>   .Select(p => new { Name = p.FirstName + " " + p.LastName, 
> Birthdate = p.Birthdate })
>
> It gets compiled to the following SQL:
> -- Region Parameters
> DECLARE @p0 DateTime = '1989-05-01 00:00:00.000'
> DECLARE @p1 DateTime = '1999-05-01 00:00:00.000'
> DECLARE @p2 NVarChar(1000) = ' '
> -- EndRegion
> SELECT ([t0].[FirstName] + @p2) + [t0].[LastName] AS [Name], 
> [t0].[Birthdate]
> FROM [Person] AS [t0]
> WHERE ([t0].[Birthdate] > @p0) AND ([t0].[Birthdate] < @p1)
>
> As you can see the mapping and filtering is done entirely on the
> database server side. The only magic need is to make the 
> compiler to
> dump the AST. In C# that's accomplished by wrapping the function
> type F in to an Expression<F> [0]. For example C#'s analog of:
>
> InputRange!T filter(T)(InputRange!T source, bool delegate(T) 
> predicate)
>
> is expressed as:
>
> IEnumerable<T> Where<T>(IEnumerable<T> source, Func<T, bool> 
> predicate)
>
> In order to request the AST of the predicate function, it needs 
> to
> be wrapped in Expression<T>:
>
> IQueryable<T> Where<T>(
>     IQueryable<T> source, Expression<Func<T, bool>> predicate)
>
> (IQueryable<T> [1] is an extension of IEnumerable<T> [2] 
> interface (which is
> similar to D's Input/ForwardRange-s) which adds the Expression 
> property
> which represents the AST of the query against the IQueryable 
> data source.)
>
> ----
>
> In addition to compiling range operations to database queries, 
> this would
> also be useful specializing on lambda's in range libraries (see 
> [3]) and
> much more.
>
>
> [0]: 
> https://msdn.microsoft.com/en-us/library/bb335710(v=vs.110).aspx
> [1]: 
> https://msdn.microsoft.com/en-us/library/bb351562(v=vs.110).aspx
> [2]: 
> https://msdn.microsoft.com/en-us/library/9eekhta0(v=vs.110).aspx
> [3]: https://github.com/dlang/phobos/pull/4265