[dmd-internals] dmd commit, revision 657

[Don Clugston]

>>  DMD's test suite is monotomically increasing in completeness and releases
>> don't comment out tests except in some extreme exceptions.. such as
>> rolling back a change due to finding regression that wasn't covered by the
>> test suite.
>>
>> ie:
>>  fix a bug + add a test that showed the bug
>>  discover a bug that fix caused that didn't have a test in the suite
>>  roll back the fix and the new test
>>  re-fix with both the new test and the newly discovered test
>
> It should be simpler than that... When a bug is discovered, add a test. In your example above, two tests should be committed regardless of if the attempted patch is committed.
>
> Let's take a hypothetical case of that Don attempted a patch and the new bug is best handled by Walter. The new tests represent a very clear communication of the problem space. Done right, that should make it easier for Walter to repeat Don's tests.

> Similarly, many bugzilla entries gave a "reduced test case" posted to them when Don confirms the bug. Such tests should also make their way into the test suite.

That could be helpful, and save a little bit of time. Note, though,
that usually, by the time I've created a good test case, I've nearly
fixed the bug.

> I have no idea how Don currently concludes that patch for bugzilla 1234333 also fixes bugzilla 333322 and 3462663. A test suite with failing tests is one way to determine such things.

I think the benefit would only be modest, since the test suite only
works for bugs which have been reduced - and generally the number of
unfixed reduced test cases is only a couple of dozen.

But I do agree that a public test suite opens a lot of opportunities
for making things easier and more efficient for everyone. Potentially,
we could remove the need for Walter to touch it at all, except in rare
cases. That would be a huge win.