D needs to publicize its speed of compilation

[Atila Neves]

On Thursday, 28 December 2017 at 21:58:07 UTC, Walter Bright 
wrote:
> On 12/28/2017 5:22 AM, Atila Neves wrote:
>> On Thursday, 28 December 2017 at 07:21:28 UTC, Walter Bright 
>> wrote:
>>> On 12/27/2017 1:23 PM, Atila Neves wrote:
>>>> However, my experience has been that D has fast builds from 
>>>> scratch, but is really really slow for incremental builds.
>>> You can do fast incremental builds with D by not putting all 
>>> the files on the command line. Just put one.
>> 
>> I don't build the compiler command-line myself, nor do I want 
>> to. Even then, recompiling one file isn't useful to me, what 
>> is is to recompile what's needed and run the tests. 
>> Emacs/flycheck already highlights compilation errors in just 
>> one file.
>
> I don't understand. C++ compiles files one at a time. You can 
> do the same in D. How can this model be useful in C++ but not 
> with D?

Because:

. As Iain pointed out independently, D modules are like C/C++ 
headers. All dependencies must also be recompiled. If I edit a 
C++ source file, only that source file has to be recompiled. I 
can use .di files, and they can even be generated for me (nice!) 
but there's no build system to do this properly. This is one of 
the ideas I alluded to earlier - I want to put this sort of thing 
in reggae. And now that dmd is a dub package I can even just use 
the actual compiler code. i.e. Generate the .di, if it's the same 
as before don't overwrite the old one to not mess with the 
timestamp.

. C++ doesn't have built-in unit tests. Which means a framework 
has to be used and since nobody wants to put their tests in 
#ifdef "blocks" they live in separate files. Changing the 
implementation doesn't require the tests to be rebuilt. Yes, I 
can separate them in D too, but then I lose DDoc benefits and 
it's actually nicer to see tests next to the code. The fact that 
all D files are effectively headers means editing a unittest 
means recompiling all modules that depend on the current one.

. Related to the previous point: compiling separately in D is 
actually slower overall than by package, because of all of the 
dependencies (I measured). Yes, I can compile by package, but I 
don't want to do this by hand. To the extent of my knowledge the 
only way to get per-package compilation by default or otherwise 
in the D ecosystem is reggae. But:

. Until recently, it wasn't possible to use 
__traits(getUnitTests) with separate compilation. I've run into 
other issues with when doing so (I can't remember what they 
were). Even with the bug fix editing a test entails compiling 
several modules and it's slower than just one C++ file. In C++ 
editing a test source file always means recompiling exactly one 
file.

. Speaking of dependencies, there's no easy way currently to know 
what they are. _If_ I want to build per package/module and only 
build what's needed, then I need to either hand-roll a Makefile 
(points gun to head) or use reggae. If there's any other 
alternative, I don't know about it. Even Phobos recompiles the 
world. That's not a good example to set.

. C++ test framework's hacky usage of global variable assignment 
to register tests seems to be faster to compile than D's 
compile-time reflection to achieve the same result. I'd have to 
benchmark to be sure though.

. If one has dub dependencies (and in all likelihood one has), 
then either one is a reggae early adopter (I keep mentioning it 
to highlight why I wrote it in the first place), or in the 99.9% 
of other users typing either `dub build` or `dub test`. There is 
no per-package build option. There's a per-module option, but 
it's single-threaded, therefore slow. Also, no dependency 
tracking, so let's recompile the whole dub package! In one 
thread. Zzzzzzzzz.

What I really want is to hit F5 in emacs, rebuild the current 
file _and_ all of its dependencies, cache the dependency build 
per file so that if they haven't change don't rebuild, link and 
run the unit tests. But build _only_ what's needed to avoid 
linker errors in order to be able to run the unit tests for _one_ 
file.

I spent an afternoon writing the necessary elisp to get this done 
and it worked! Then I tried a different project and it failed 
miserably. So did the 3rd one. I think I just remembered one of 
the separate compilation bugs I mentioned above. I know, I know, 
if I don't file it it won't get fixed (even if I file it I have a 
sneaky suspicion it'll be me fixing it anyway), but figuring out 
what went wrong takes time and energy and I haven't had enough of 
either to take this off my TODO list.

To summarise:

. Dependencies.
. There's no standard build system that isn't called dub.
. There's only one alternative build system that can do any of 
what's needed.
. That alternative build system hasn't implemented my .di idea 
yet.
. Separate compilation bugs. That nobody ever hits because dub 
builds everything at once.



Atila