Appreciating D

[IchorDev]

This is going to be a long one.

I have been thinking about [this blog 
post](https://www.yet-another-blog.com/porting_the_game_to_jai_part0/) a lot recently. It documents how the author—now presumed dead—first used D in 2019 as a replacement to C++ for programming video-games. They discuss how they didn't like the experience of using D, and why they chose to port their code to Jai, an unfinished language which is not available to the general public. They neglect to mention the future implications of the language's unfinished unreleased state for libraryavailability, however.

In the present day it's only nearly two years later, Jai has 
still not materialised a public release despite its seemingly 
avid community, and around half of the problems with D mentioned 
in the post have already been solved. I think this is a great 
indication of the strength of D's community. Despite some recent 
roadblocks in getting new features into the language, it has 
still evolved significantly over the course of a mere two years. 
I'd like to now go over some of the issues the author had with D 
at the time and review them for the sake of appreciating D's 
growth, and analysing some of their misconceptions about the 
language.


> After bouncing between the two available D compilers for 
> Windows (dmd and ldc2) for about 4 years, I find the state of D 
> on Windows to resemble something I would expect from a hobby 
> project, [...]
> From the following problems that I have encountered over the 
> years, by far the biggest one is that debug info on Windows is 
> completely broken:
> [...]
> I was told that “DMD has historically had a lot of issues with 
> [...] debug info” [...] but unfortunately most of [these 
> issues] apply to both compilers, not just dmd.

First of all, GDC is also [available for 
Windows](https://winlibs.com), at least in some capacity.
I cannot speak for the quality of D's compilers on Windows (since 
I have banned it from my house), however I can say that since 
they experienced issues with LDC2 I suspect that a lot of their 
issues with debugging could stem from Visual Studio, or how it 
interfaces with whichever debugger it's using. We're never 
informed whether it was VSCode or the full VS IDE though, so the 
answer remains nebulous.

> - mixins (D’s macro equivalent) generate debug info in a way 
> that causes the debugger to not find the correct file (so you 
> step through disassembly)

The `-mixin` flag has always solved this problem for me, and I 
don't see why it wouldn't work in Windows.

> [...] there are other issues and shortcomings, to a significant 
> part in metaprogramming:
> - different compiler phases interact in weird ways that lead to 
> surprises in metaprogramming, while generating misleading 
> errors [...] like order independent declaration in global 
> scopes sometimes breaking with `mixin`s

I feel that this is to be expected, and this behaviour can be 
easily logically identified with [a bit of 
knowledge](https://dlang.org/spec/intro.html#phases-of-compilation). A compiler is not magical, it cannot know things it has not gone out and discovered yet. Usually D doesn't require forward declaration because it does a pass that alleviates the need for it, but without giving mixins their own pass, they naturally result in sequentially parsed code. D's backend design has a strong preference for shorter compile times, and adding another pass would be a huge detriment to compile times.
D compilers definitely have an error message problem, though. 
Since this post was written many previously confusing error 
messages (especially for missing semicolons & closing 
parenthesis) have gotten a lot better, but you'll always get 
insane error messages from the depths of a fever dream when the 
compiler encounters a fringe error (or possibly even a bug) 
inside of a 7-layer nested mixin from a CTFE lambda inside an 
instantiation of an aliased template.

> or like namespaced names not resolving in mixins unless you use 
> a [special kind of string 
> literal](https://dlang.org/spec/lex.html#token_strings)

This is a very strange one. First of all, D does not really use 
'namespaces', instead we use have scopes. What I *assume* they 
mean is fully qualified identifiers, but I cannot find any 
evidence for anything like this in the history of the D 
specification for mixins or token strings. The other possibility 
is that they're using D's 'namespace' variant of `extern(C++)` 
(which I do not recommend—use the string version instead), since 
they mention C++ interoperability. Perhaps one of those was once 
an implementation detail, but it's also a patently illogical 
idea. Mixins parse strings into a regular D 
expressions/declarations, so why would they ever have a special 
case like this?

> or like `static foreach` not being able to insert multiple 
> `else if`s after an `if`

Here we see a fundamental misunderstanding of D's 
meta-programming. The body of a `static foreach` inlines a series 
of [Declaration 
Statement](https://dlang.org/spec/statement.html#declaration-statement)s. None of D's meta-programming facilities allow you to write incomplete declarations/expressions the way that C's macros do, because of the disastrous effects this has had on C's readability. If you want to write incomplete declarations like having `if` and `else if` declared with meta-programming, use string concatenation with a mixin:
```d
void main(){
	int n;
	import std.writeln;
	mixin((string[] list){
		string ret = "if(n == "~list[0]~"){\nwriteln(`"~list[0]~"`);\n";
		foreach(item; list[1..$]){
			ret ~= "}else if(n == "~item~"){\nwriteln(`"~item~"`);\n";
		}
		return ret ~ "}";
	}(["1", "2", "3"]));
}
```

> ldc2 is awfully slow to compile:
> It takes 1 minute in debug and over 5 minutes in release mode 
> to compile the game. There’s also the issue of ldc2 somehow 
> needing 8GB of RAM in debug and 11.5GB in release mode for 
> compiling a 20MB executable from 2MB of source files, thus 
> causing regular near-death experiences for my laptop.

When hearing about compile times this slow, and using this much 
memory, the first thing that comes to mind is **severe** template 
abuse, which is a compile time bottleneck I've experienced with 
LDC2 before, especially when building for release. What's the 
solution? First of all, acknowledge that whenever you use 
meta-programming, you're making the compiler **run** that code, 
so if that code takes a long time to execute then that will 
necessarily inflate your compile times. No compiler design can 
fix that. A big part of the issue for me was re-instantiating 
template functions in nested `static foreach` loops. Instead, I 
would recommend using CTFE functions that create one big mixin.

> but [ldc2 is] sometimes the only choice because dmd has bugs:
> I’ve encountered a couple of bugs over the years that stopped 
> my program from building, the latest one being this [codegen 
> bug](https://issues.dlang.org/show_bug.cgi?id=23195) when 
> interfacing with C++.

The bug mentioned has been [fixed for some time 
now](https://github.com/dlang/dmd/pull/14651#event-8476344353), 
and importantly, was introduced due to an error in Microsoft's 
own documentation about the calling convention.

> D offers a *betterC* mode that among other things disables 
> garbage collection. However, when using this mode, the standard 
> library does not compile and meta programming is significantly 
> hampered:
> Disabling the garbage collector not only disables it for your 
> compiled code, but also for the code executing at compile time. 
> One major way of metaprogramming in D is injecting new code 
> into the program. That code is a string, and those strings need 
> to be concatenated, and string concatenation uses the garbage 
> collector. So … it just doesn’t compile.

There is quite the errata in this section. Specifically, BetterC 
replaces DRuntime with a wrapper that hooks into C's runtime, 
which makes some basic DRuntime-dependant features like `assert` 
still work. The GC is part of DRuntime, so out it goes. 
Phobos—D's standard library—is *partially* unavailable, with most 
meta-programming facilities (like `std.traits`/`std.meta`) still 
being available.
You might've noticed a discrepancy there. Why would removing the 
garbage collector from the *runtime* stop GC being used at 
*compile-time*? The compiler's interpreter isn't using the same 
runtime as our generated run-time code, right? No, of course it's 
not. D compilers don't even use the GC at compile-time by 
default, although it can be enabled for lower memory consumption. 
What this person has clearly encountered is that run-time 
functions that use the GC/DRuntime naturally don't compile with 
BetterC (or produce linker errors) and so they have naturally 
concluded that CTFE string concatenation is impossible with 
BetterC. This is simply wrong. When you write a run-time function 
(**whether or not** you use it in run-time code) the compiler 
**will** try to generate code for it. There's a way to make 
CTFE-only code-paths in a function, but this feature only works 
with `if`, not `static if`, so it's not useful to us here. What 
you need to do is create a function that the compiler won't try 
to generate run-time code for. You need a lambda:
```d
//not a lambda, will generate run-time code:
string catRT(string a, string b) => a ~ b; //Error: array 
concatenation of expression `a ~ b` requires the GC which is not 
available with -betterC

//a lambda assigned to an enum:
enum catCT = (string a, string b) => a ~ b; //OK
```
Of course, if you call this lambda in run-time code it will 
produce an error (from linking, specifically).

> Bad debug info and the de-facto need for garbage collection are 
> dealbreakers

Garbage collection being a deal-breaker is weird to me as a D 
user, but somewhat understandable for real-time speed-critical 
applications. However, D is a perfectly capable language without 
DRuntime, as you can easily write a custom templated 'array' 
struct that overloads `~` and uses existing C/C++ libraries 
instead of Phobos. If you don't care about executable size, just 
mix GC with manual memory allocation. You'll get to use a lot 
more D libraries that way. Unless you pick a good allocator or 
swap a lot of large long-term GC allocations to manual ones (to 
reduce GC heap scanning) your net time savings will likely be ~0 
anyway.

See this excellent write-up about the matter from Bit Bashing for 
more: [Garbage Collection for Systems 
Programmers](https://bitbashing.io/gc-for-systems-programmers.html)

> **Why is there a single language without the ability to pass 
> parameters by name in 2022??**

The [DIP for named 
parameters](https://github.com/dlang/DIPs/blob/master/DIPs/accepted/DIP1030.md) was accepted back in 2020, so anyone could've seen it coming, but now we've finally got our hands on them! Hurrah!

And that's all! If you like long blog posts about D I recommend 
reading some from [Bit Bashing](https://bitbashing.io), and [The 
Art of Machinery](https://theartofmachinery.com).