Reimplementing the bulk of std.meta iteratively

[Paul Backus]

On Tuesday, 29 September 2020 at 03:14:34 UTC, Andrei 
Alexandrescu wrote:
> D breaks that principle in several places. It has had a 
> cavalier attitude to using magic tricks in the compiler to get 
> things going, at the expense of fuzzy corners and odd limit 
> cases. Look at hashtables. Nobody can create an equivalent 
> user-defined type, and worse, nobody knows exactly why. (I 
> recall vaguely it has something to do, among many other things, 
> with qualified keys that are statically-sized arrays. Hacks in 
> the compiler make those work, but D's own type system rejects 
> the equivalent code. So quite literally D's type system cannot 
> verify its own capabilities.)
>
[...]
>
> I very strongly believe that this complexity, if unchecked, 
> will kill the D language. It will sink under its own weight. We 
> need a precise, clear definition of the language, we need a 
> principled approach to defining and extending features. The 
> only right way to extend D at this point is to remove the odd 
> failure modes created by said cavalier approach to doing 
> things. Why can't I access typeid during compilation, when I 
> can access other pointers? Turns out typeid is a palimpsest 
> that has been written over so many times, even Walter cannot 
> understand it. He told me plainly to forget about trying to use 
> typeid and write equivalent code from scratch instead. That 
> code has achieved lifetime employment.

I think D is already too deep in the tar pit to ever fully 
extricate itself. All of those special cases and "magic tricks" 
are there because, as you say, code depends on them to work. 
Getting read of them would break the world a hundred times over. 
If you're going to do that, you might as well start a new 
language from scratch, no?

The best we can hope for is to nail down a precise description of 
all the fiddly corner cases, so that we know what D's semantics 
*actually* are, rather than what the current spec pretends they 
are. But even that is likely to be infeasible in practice--who 
wants to volunteer to be the one to go over, say, 
funcDeclarationSemantic with a fine-toothed comb, especially 
after what happened to the last guy? [1]

The fact is, there's no incentive for anyone to do this kind of 
work other than masochism and sheer stubbornness. The amount of 
effort it would take, and the amount of time that would pass 
before you saw any benefit, are simply too high. We can talk 
about principles and good engineering all we want, but we can't 
expect people to voluntarily act against their own interest.

Maybe we can make some improvements at the margins, but my 
prediction is that the overwhelming majority of D's complexity is 
here to stay, whether we like it or not. The best we can do is 
try to offer enough positive value to D programmers to offset its 
negative effects. And if it doesn't work out, well, there's 
always Rust. ;)

[1] https://github.com/dlang/dmd/pull/8668