Compilation is taking a ton of memory

[crimaniak]

On Wednesday, 27 June 2018 at 16:00:37 UTC, Mario Silva wrote:
> Hello,
>
> Our code base has been growing steadily and it's currently at a 
> point where my 16GB machine just freezes when we're compiling 
> our code. This happens because DMD just consumes all my memory 
> for a while.
>
> Also, it's taking a long time to compile it. Less than an year 
> ago our project was taking around 17 seconds to compile - no 
> libs requiring compilation - and maybe around 50 seconds for 
> full compilation, and it now takes around 50 seconds for an 
> incremental compilations and around 1.5 minutes for a full one.
  The same problem. Freeze because of active swap usage and about 
2 minutes to compile after every small change. In fact, when it 
comes to template code, DMD is neither particularly fast nor 
memory-efficient. Given that there are no things such as 
precompiled headers, on a large project, the D experience is much 
worse than C++.
The problem is aggravated by the fact that DUB compiles all the 
sources in one DMD launch. I tried to solve this problem some 
time ago, and Sönke advised using reggae 
(https://code.dlang.org/packages/reggae). I'm still experimenting 
with different approaches and even wrote my own utility helper 
(https://code.dlang.org/packages/ifupdated). But at the moment I 
can state the following:
  * It is necessary to incorporate into the DUB reggae (== classic 
make) approach to compile sources.
  * Like the 'debug' and 'release' modes of the compiler, the 
build system should also have two modes, for example, 'deploy' 
and 'development', the first minimizes the time it takes for a 
full compilation, and the second minimizes the recompilation time 
after minor changes.
  * It needs to think seriously about 'compiler as a service' 
approach and caching intermediate compilation results.

> Any tips on how to code in a way that minimizes both 
> compilation times and memory consumption when compiling?
  For the current moment just try reggae. You can find some 
forgotten imports because of separate compilation, but it is 
easily fixable.