[dmd-internals] Windows DMD development is now impossible

[Don Clugston]

On 10 June 2013 11:25, Martin Nowak <code at dawg.eu> wrote:

> On 06/10/2013 08:29 AM, Don Clugston wrote:
>
>> Yeah, my point was that for N tests, we get O(N^^2) templates
>> instantiated.
>>
> How is that?


Just correlation, not causality! The million-ish instantations only comes
from a thousand tests.

> Wow! (How many of those are unique, rather than reusing an existing
instantiation?)

The number of instances is also a huge performance issue when searching for
> existing instantiations, because search is done linear and comparing two
> TemplateInstance uses the costly arrayObjectMatch on the template
> arguments. So in average instantiating the same template N times uses
> O(N^^2 / 2) comparisons.
> I once improved that by computing the mangling before the search and using
> it as a key.
> That brought down the compilation time of std.range with unittests from 10
> to 2 seconds.
> There is at least one blocker though http://d.puremagic.com/issues/**
> show_bug.cgi?id=7469 <http://d.puremagic.com/issues/show_bug.cgi?id=7469>.
>

Interesting. This is looking like an area where an order of magnitude
improvement in both memory consumption and speed is possible...


> Error: out of memory
>
> > This has happened many times before, and we dealt with it by reducing
> the number of modules we compiled into each object file. We once had 30
> modules per obj file. Then fifteen. Then five. But now we're at one, that
> workaround can no longer be used.
>
> I ran into some issue when preparing the phobos unittests for shared
> libraries.
> If you compile the whole library at once with unittests enabled it
> consumes ~6GB of RAM.
> In order to find a good partition I took up the idea of clustering (
> http://d.puremagic.com/issues/**show_bug.cgi?id=9673#c2<http://d.puremagic.com/issues/show_bug.cgi?id=9673#c2>,
> https://gist.github.com/**dawgfoto/5747405<https://gist.github.com/dawgfoto/5747405>).
> Well, at least clustering by imports did only find a few useful partitions
> because almost every module ends up importing std.range or std.algorithm.
> It might be interesting to do some analysis for the template instances used
> in each module.


I think the biggest problem is traits.d.  It contains ultra-lightweight
templates that are used everywhere and have no dependencies, things like
Unqual!(). But it imports
std.typetuple (which is OK, it only imports std.traits),
std.typecons. (which imports std.array, std.conv, std.exception,
std.format, std.string, std.range)
std.algorithm (which imports std.array, std.ascii, std.container, std.conv,
std.exception, std.functional, std.math, std.random, std.range, std.string,
    std.uni, std.utf)
... ie, basically all of Phobos

If we could split those things out into a 'leaf' module, probably most of
the import graph complexity would disappear. I'm sure most of the cyclic
imports are false dependencies.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/dmd-internals/attachments/20130610/44875e4b/attachment.html>