Running out of memory ctfe 16GB

[Dmitry Olshansky via Digitalmars-d]

On 4/6/17 10:49 PM, Nierjerson wrote:
> I am running out of memory trying to generate CTFE code. It is quite
> large generation but just repetitive loop acting on an array item.
>
> Surely 16GB should be enough to compile such a thing? I am using 64-bit
> dmd. It climes to about 12GB then eventually drops down to around 1GB
> and then back up to 16GB and then quits.

I know how it feels.
CTFE allocating lots of objects and never bothering to free them is a 
known problem that has plagued D for years.
The new hope is the NewCTFE engine that ought to be reasonable on memory 
footprint and actually faster as an interpreter.

>
> At the very least have something to give feedback on how to reduce
> memory consumption. Leaving things up in the air for programmers to
> stumble upon after a lot of work is not good.

I just did a cursory look on the source but a few things you may try:

1) Use plain string instead of wstring for codegen, since this is simply 
source code which is ASCII I don't see a need for UTF-16. Technically 
should reduce the size of arrays involved x2.

2) Instead of building out out big string try building each interface 
separately. If I'm not mistaken current CTFE will actually allocate a 
whole new array on each append and copy things.
Since it never deallocates building up a huge string by bits and pieces 
is a bad idea as it will leak the entire thing on each append.


----
Dmitry Olshansky