DMD performance issues with deeply-recursive templates

[H. S. Teoh]

I just noticed this bug today:

	http://d.puremagic.com/issues/show_bug.cgi?id=10693

>From what I can tell from a cursory glance, it's caused by the
implementation of cartesianProduct using too many template expansions
(esp. in the variadic version of cartesianProduct), causing DMD to run
very slowly and eventually exhaust all available memory and crash.

It's not too difficult to rewrite cartesianProduct to use less
templates, of course, but the reason I wrote cartesianProduct the way I
did in the first place was because I wanted to use it to prove that
functional style programming and deep range composition in D is viable.
So I didn't write anything from scratch, but just put existing ranges
together to produce the desired result.

However, now I'm starting to wonder if we need to be taking a look at
how DMD handles template expansions to see if we can reduce the memory
footprint as well as compile-time performance. Or perhaps whether we
need some way of managing combinatorial explosion when it comes to
expanding recursive templates.  Maybe it's also time for us to start
thinking about how we might optimize deeply-composed ranges -- perhaps
with some compiler built-in help?

Or are we stuck with needing to implement Phobos ranges from scratch in
order to have acceptable compile performance (and template bloat
minimization), because dogfooding is simply not viable with the current
state of things?


T

-- 
"The number you have dialed is imaginary. Please rotate your phone 90
degrees and try again."