[gsoc] DMD - Ideas

[Mike Franklin]

On Thursday, 7 March 2019 at 09:53:13 UTC, Seb wrote:
> @community: what other features do you miss the most from DMD 
> that could be done in the span of a GSoC?

Convert runtime hooks [1] to templates

Example
--------
void main()
{
     auto bArray = new byte[5];
     bArray.length = 8;  // gets lowered to `_d_arraysetlengthT`
}

`bArray.length = 8` gets lowered by the compiler to the runtime 
hook `_d_arraysetlengthT` [2].  If you look at the signature for 
`_d_arraysetlengthT`, you'll see that it takes a runtime 
`TypeInfo` object. That is unfortunate because everything the 
compiler needs for the implementation is known at compile-time. 
Therefore, `_d_arraysetlengthT` should be converted to a 
template, and the compiler should be changed to lower 
`bArray.length = 8` to the template implementation.

Lucia Cojocaru led the charge on this in 2017.  She gave a great 
talk about it at DConf 2017 [3].  Her motivations for doing this 
work came accross much different than mine, but still valid and 
beneficial to D.  For me, one of the problems with `TypeInfo` is 
that it is a class, and clesses require a great deal of runtime 
support in D.  If the runtime hooks have a dependency on 
`TypeInfo` classes, then that means every D language feature that 
gets lowered to a runtime hook also depends on runtime support 
for classes.  Runtime support for classes includes the GC, and 
that doesn't scale well to bare-metal and resource-constrained 
platforms that neither want, nor need, that runtime overhead.  
Implementing the runtime hooks as templates leverages D's 
all-star features such as design by introspection, 
metaprogramming, and CTFE which can yield performance benefits as 
demonstrated by Lucia.

Although it was before my time, my understanding for why things 
are currently implemented with runtime `TypeInfo` when everything 
is known at compile-time is because the runtime hooks were 
written before D had templates. Now that D has templates, it 
seems, at least to me, that templates is the obvious choice.

There are challenges with this effort, however. In the compiler, 
the lowerings are performed after the semantic phase in e2ir.d 
[4]. This means that currently, for example, setting the length 
of a dynamic array can be done in a `@safe`, `pure`, and 
`nothrow` context [5], but `_d_arraysetlengthT` is neither 
`@safe`, `pure`, nor `nothrow` [2]. The compiler is lying to us. 
So, if you convert these runtime hooks to templates, the lowering 
will be done in the semantic phase, and suddenly, you'll be 
forced to be honest. After a conversation with Walter, I was told 
to make use of `pureMalloc` and friends [6] to work around the 
`pure` constraint. `trusted` can be used to work around the 
`@safe` constraint, and instead of throwing exceptions in 
`nothrow` code, `assert`ions can be used.

See [7] for an example implementation.

Mike

[1] https://wiki.dlang.org/Runtime_Hooks
[2] 
https://github.com/dlang/druntime/blob/c86e4a0e541dc63f9e9f2ee09c7efd325c0be2d5/src/rt/lifetime.d#L1443
[3] https://www.youtube.com/watch?v=endKC3fDxqs
[4] 
https://github.com/dlang/dmd/blob/f4f29c3e47ee30b2646cede35ed0f9bd823f0add/src/dmd/e2ir.d#L2611
[5] https://run.dlang.io/is/2aefNu
[6] 
https://github.com/dlang/druntime/blob/29f495b5b3571484bcf4d0af2fe211d6b7d86830/src/core/memory.d#L862
[7] https://github.com/dlang/dmd/pull/8531