Build time

[H. S. Teoh]

On Fri, Jul 23, 2021 at 06:53:06PM +0000, JG via Digitalmars-d-learn wrote:
[...]
> The program I writing is around 3000 loc and recently I noticed a
> large slow down in compile time which after investigation seemed to be
> caused by my computer running out of memory. The compile was using
> more than 15GB memory.  I tried using lowmem and that did solve the
> memory problem but the compile still takes around 1 minute. Any
> suggestion on how to try and improve the build time. I am currently
> using dub.

3000 loc and 1 minute build time?  Sounds like you're using too many
nested templates / CTFE.


> Of course one could try to use fewer templates and less meta
> programming but that seems to defeat the purpose of using d.

I wouldn't say use fewer templates / less meta-programming.  But I'd say
look into how deeply nested templates are, and whether some templates
parameters may be unnecessary.  If you have recursive templates,
consider refactoring it so that it uses linear expansion instead.
Shallowly-nested templates generally don't run into performance
problems.

(Confession: I wrote the variadic version of cartesianProduct in
std.algorithm with recursive templates. It uses an exponential number of
template expansions, and so quickly brings the compiler to its knees
when you try to take 4 or more cartesian products in a row.  Eventually,
I refactored the most common case (no infinite ranges among its
arguments) to use a linear expansion with a nested loop instead.
Compile times improved by a HUGE margin.)

And avoid doing too much work in CTFE, which is known to be slow. But
not as slow as overly-deeply nested templates.

Another way is to have a separate build step for expanding the most
heavy templates, so that you only incur that heavy expansion once in a
while when you change the relevant code.  I had a Vibe.d project where
Diet templates slowed me down too much (they are super template-heavy),
so I split it into several different build targets with a separate link
step, so that when I'm not changing the Diet templates it doesn't slow
me down so much.  Dub unfortunately won't help you here (unless you use
subpackages -- but I doubt it will win much) -- I recommend using a
better build system like CMake or SCons. Dub's architecture simply does
not play well with staged compilation.

Alternatively, use a separate pre-compilation stage for generating code
(e.g., write a utility that emits D code that then gets compiled in a
subsequent step).  As much as I love D's compile-time capabilities,
there comes a time when it's simply more practical to just `writefln`
some D code snippets into a file and compile that into the main program,
instead of trying to tame the memory-guzzling beast that is the D
compiler.


T

-- 
I am Ohm of Borg. Resistance is voltage over current.