[phobos] Time taken for running unit tests

[Don Clugston]

On 28 September 2011 10:30, Jonathan M Davis <jmdavisProg at gmx.com> wrote:
> On Monday, September 26, 2011 11:37:30 Don Clugston wrote:
>> I think we need to have a strategy for managing the amount of time
>> required for running the unittests. Currently, on Windows, the time
>> for all compiler tests + phobos tests is one hour.
>
> Wow. Your machine must not be particularly new.

The one hour is phobos tests *plus dmd tests*, not just the Phobos tests.
That's the timing from the autotester.

> Interestingly enough though, the dmd build that I had on Windows initially
> (which was from towards the beginning of this month) before I updated today
> took just under 6 minutes. So, the changes since then (probably the GC stuff)
> made it take about a third longer than before.

Yes. It seems like a cure that's far worse than the disease.

> Regardless, dmd's performance obviously needs to be improved, and there would
> be a significant speed gain in redoing the Windows unit test build to be split
> up like the Linux build.

Yes. But, even so, we're still going to run into this issue sooner or
later. For example, the test that Andrei recently shortened because it
was taking so long, was instantiating almost 1000 templates, despite
being a quite reasonable test for a single function. If all Phobos
functions had similar tests, we're already talking about millions of
template instantiations.
When you think about what would be required to thoroughly test the
FreD regex engine, it gets even worse. Sure we can make the compiler
much faster, but the numbers are intrinsically huge.