Faster linker

[Steven Schveighoffer]

On 11/8/18 9:20 PM, H. S. Teoh wrote:
> On Fri, Nov 09, 2018 at 01:04:51AM +0000, bennygui via Digitalmars-d wrote:
>> Hi,
>>
>> I am compiling a small vibe.d project with dmd via dub. The linker
>> takes most of the time when invoking "dub build" (total: 11 seconds).
>> I've tried compiling with gdc and it's the same (11 seconds). With
>> ldc, it's faster (6 seconds).
>>
>> I did not time anything except "dub build" but I get the feeling that
>> dmd compiles much faster but is slowed down by the linker and that ldc
>> takes longer to compile but has a faster linker.
>>
>> Is there any way to get faster link time? I'm on Linux (Gentoo).
> [...]
> 
> Are you sure it's the linker, and not the compiler, or dub itself?

Yeah, a vibe.d project with diet templates takes most of its time in the 
linker, I've found. Basically a big hang after dub prints:

Linking...

> If it's the linker, then you could look into using gold instead of the
> BFD linker.
> 
> I suspect most of your build time is spent in dub, though.  You could
> try various options to turn off network access and dependency resolution
> when running dub, which should improve total running time. (Of course,
> you need to run it at least once without those options so that it pulls
> in required dependencies, but once those are downloaded, there's no need
> to check dependencies every single time.)  You could also ditch dub and
> use your own build system, by creating an empty dummy project in a
> subdirectory and running dub once to pull in your dependencies, then
> link to the artifacts yourself directly. (I did that in one of my own
> projects, and it sped up build time by an order of magnitude.)
> 
> Another possible cause is that Diet templates are very template- and
> CTFE-heavy, and will often soak up gobs of RAM and CPU every time they
> are compiled.  A possible solution is to separate out the Diet templates
> into their own source files, and use separate compilation to avoid
> rebuilding those files unless there was an actual change in the Diet
> code. (I've also run into this in my vibe.d project. I'm considering to
> replace Diet templates with something lighter-weight, but haven't taken
> that step yet.)

I suspect the issue is not exactly the linker, but the symbol name sizes 
again (probably related to the diet templates). I haven't looked into it 
enough to see where the problem is coming from, but one of either vibe.d 
using naming tricks to limit the name size, or D doing something clever 
with string template parameters should help.

-Steve