D is our last hope

[Adam D Ruppe]

On Thursday, 21 December 2023 at 23:04:30 UTC, Walter Bright 
wrote:
> It's a good point. One of the likely suspects is object.d.

object.d is not significantly to blame.

First, important to establish a baseline. Just make an empty file 
and compile it:

$ time dmd -c empty

real    0m0.011s

Even if you brought the import time of object.d to absolute zero, 
you'd have *at most* 10ms to gain from it. This is less than 5% 
of the test times.

Even if you do a compile and link:

$ time dmd empty -main

real    0m0.163s

Most this time is spent in the linker, not the compiler. 
druntime's pre-compiled lib could potentially be smaller to 
reduce link times, but changing object.d will do very little for 
this.

> have to be compiled by every separately compiled D module.

Secondly, even if it were true that object.d were to blame, this 
would be an incorrect path. object.d is never *compiled*, it is 
merely imported.

Which means:

> For example, object.d has unit tests in it.

This is irrelevant. The unit tests are skipped with extreme 
rapidity. So are function bodies. The things the .di process 
strips out are already automatically skipped by the existing 
import process!

I went into more detail recently here:
http://dpldocs.info/this-week-in-d/Blog.Posted_2023_10_09.html#ctfe-caching

> Another culprit is writefln(). For all its faults, printf() is 
> compact, fast, does not allocate memory, and:

This is a more fruitful line of attack, but anyone looking into 
this, remember to be empirically driven. Test your assumptions - 
you'll probably be surprised at what actually makes a difference.

> Another problem is nobody ever profiles dmd anymore. There are 
> no instructions in build.d in how to run the profiler.

...but as you collect data, make sure it doesn't lead you astray. 
I remember when the dmd profiler said freeing memory was a 
significant cost, so everyone just stopped trying. Memory usage 
ballooned... and there was no actual impact on speed in the real 
world, because it turns out the specific `free()` implementation 
in the profile had a bug that no other implementation shared. 
Simply building dmd with the ldc compiler and using the Microsoft 
and GNU  C libraries completely erased that difference.