[dmd-internals] dmd commit, revision 657

[Brad Roberts]

On Thu, 2 Sep 2010, Jason House wrote:

> Here's how I'd rank things by level of dislike:
>  ? A crippled test suite where failing tests are deleted
>  ? A crippled test suite with failing tests are commented out
>  ? A complete test suite that always fails due to bugs way down on the priority list
>  ? A complete test suite that ignores tests but reports that it's ignoring tests
>  ? A complete test suite that runs all tests but is tolerant of hand-picked tests failing (xFAIL, etc...)
>  ? A complete test suite with all tests passing
> 
> The last one us unrealistic while under development. In fact, a 
> "complete test suite" is also unrealistic, but a test suite should 
> slowly become more and more thorough. Certainly, failing tests should 
> not be removed.

The last one is what we have, as long as you drop the word 'complete'.  
DMD's test suite is monotomically increasing in completeness and releases 
don't comment out tests except in some extreme exceptions.. such as 
rolling back a change due to finding regression that wasn't covered by the 
test suite.

ie:
  fix a bug + add a test that showed the bug
  discover a bug that fix caused that didn't have a test in the suite
  roll back the fix and the new test
  re-fix with both the new test and the newly discovered test

The major wish I have is that walter would start using the public test 
suite rather than the private one.  I'm working on getting the public one 
to work on windows -- making good progress, it runs most of the tests now.

Once walter actually uses that one, it'll be easier to monitor to make 
sure fixes come with tests. :)

Later,
Brad