[dmd-beta] D 2.062 beta

[Jonathan M Davis]

On Saturday, February 16, 2013 19:00:08 H. S. Teoh wrote:
> On Sat, Feb 16, 2013 at 09:49:01PM -0500, Nick Sabalausky wrote:
> > On Sat, 16 Feb 2013 18:34:22 -0800
> 
> > Walter Bright <walter at digitalmars.com> wrote:
> [...]
> 
> > > (It is a fast machine - 6 cores, 16Gb ram, it flies through the test
> > > suite!) _______________________________________________
> > 
> > Dayam!
> 
> [...]
> 
> I have a 6-core 32GB RAM machine. ;-)
> 
> Yet even I am running into out-of-memory problems when running Phobos'
> test suite in parallel. I used to be able to do it at -j6 (heck,
> sometimes I did -j8 just to max out the cores while some stuff is
> waiting on I/O), but nowadays the std.algorithm unittests for whatever
> reason take an incredible amount of memory to run -- or maybe it's DMD
> compiling the overly-gigantic std.algorithm clashing with whatever else
> is running ATM -- and even at -j2 or -j3 I risk seeing random OOM
> failures.
> 
> (But this is just my gut feeling, I didn't really look into what exactly
> was causing the OOMs. All I knew was that adding certain unittests to
> std.algorithm caused the test suite to OOM. Keeps my unittest-writing
> frenzy in check. :-P)

Anything template-heavy is going to have problems like this, and 
std.algorithm's unit tests are clearly going to be template-heavy. And the 
problem is worse on Windows, because dmd dies when it uses much more than 
900MB. IIRC, there was a patch to the backend that was supposed to increase 
the amount of memory that it could use, but I don't know if it got merged or 
not, but regardless, it obviously isn't going to use anywhere near the full 
RAM on machine with 16GB or 32GB RAM, since it's a 32-bit process.

To fix the problem, the memory consumption of templates and CTFE needs to be 
reduced, and we probably need someone to sort out the GC that dmd has which 
doesn't work very well (at minimum, making it kick in when you go above some 
amount of memory usage would be a big improvement), but no one has spent time 
on that AFAIK.

- Jonathan M Davis