Why is D unpopular?

[FeepingCreature]

On Monday, 2 May 2022 at 01:42:19 UTC, Walter Bright wrote:
> On 5/1/2022 1:39 PM, Max Samukha wrote:
>> I don't quite understand why you insist on native code. Do 
>> compilers to bytecode count? There used to be a language 
>> called Nemerle (https://en.wikipedia.org/wiki/Nemerle), which 
>> had been mentioned on these forums many times, long before D 
>> got CTFE. The earliest mention I found is 
>> https://forum.dlang.org/post/ca56h1$2k4h$1@digitaldaemon.com. 
>> It had very powerful macros, which could be used as compile 
>> time functions, AST macros, and whatnot. The language is now 
>> dead because it was too good for humans.
>
> Nemerle targeted C#'s bytecode system. I.e. it's an interpreter.
>
> A language designed for interpretation does not distinguish 
> between compile time and run time. While the program is 
> executing, it can generate more code, which the interpreter 
> executes. If the language is successful, it'll often get a 
> native code generator to speed it up. The compiler is part of 
> the runtime of the language.
>
> A language designed for native compilation draws a hard 
> distinction between compile time and run time. You'll see this 
> in the grammar for the language, in the form of a 
> constant-expression for compile time, and just expression for 
> run time. The constant-expression does constant folding at 
> compile time. The runtime does not include a compiler.
>
> D is the first language I know of, designed for native 
> compilation, with constant-expressions, that extended the 
> evaluation of constant-expressions with the ability to execute 
> functions at compile time. There's no compiler in the runtime. 
> D does not support compiling code at runtime.
>
> To clarify, if the runtime of a language includes a compiler, 
> that is in the interpreted class of languages (even if they jit 
> to native code), and it is not an example of what D's CTFE is 
> doing. Lisp, Java, C# and apparently Nemerle fall into this 
> category.
>
> To show D is not the first, I request an example of a language 
> designed for native compilation, that does not include a 
> compiler in the runtime, that has constant-expressions in the 
> grammar that must be computed at compile time and can execute 
> ordinary functions at compile time.
>
> C++ came closest to the mark with its Turing-complete templates.

This is an odd division to me. The way I do it in my lang is to 
just treat compiler runtime as a separate compilation target. 
Interpretation, code generation, it's all backend concern. But 
that still gives me the sort of "run program code at compiletime" 
capability that I think Nemerle aims at (though I've never used 
it), without any interpretation, just by targeting parts of the 
program at a backend that can be loaded back during the 
compilation run. And I think that's fundamentally a cleaner model 
than CTFE, because it doesn't rely on, in effect, embedding an 
entirely separate codepath for constant folding that reimplements 
deep parts of the compiler; instead, there is only one path 
through the compiler, and the split happens cleanly at the 
back-end ... and one backend just happens to be invoked by the 
compiler itself during compilation to get a function pointer to 
directly call.

One problem of doing it this way is that it makes `static if` 
very awkward. You're trying to evaluate an expression, but you 
want to access local "compiletime variables" - so you need to 
compile the expression being tested in an odd context where it 
lives in a function, "but not really" - any access to local 
variables triggers a special error, because the expression is 
really semantically at something like toplevel, it just 
syntactically lives inside a function. That's why for now, I just 
use constant folding like D for `static if`. (Also, I'm dog slow 
and I'm trying to limit the amount of macro compilation 
roundtrips. Gonna switch to an *interpreter* backend some time - 
for a *speedup.* LLVM makes good code, but its performance is 
painful.)

But I still think this is fundamentally the right way to think 
about CTFE. The compiler runtime is just a backend target, and 
because the compiler is a library, the macro can just recurse 
back into the compiler for parsing and helper functions. It's 
elegant, it gets native performance and complete language support 
"for free", and most importantly, it did not require much effort 
to implement.