[phobos] Silent failure of std.container unittests

[Michel Fortin]

Le 2010-07-15 à 18:00, Walter Bright a écrit :

>> 4. DESIRED: assert is NOT hacked, any failing assert ends the current unittest, the failure message is printed, execution continues with the next unittest, program ends with error code if at least one assert failed, everybody's happy.
> 
> Everybody's happy except the guys like me who want to run all the unit tests in one go.

I think this is the root of the misunderstanding. Either you consider each assert grouped in a unittest blocks to be a unit test in itself; or you consider each unittest block to be a test with multiple possible points of failure across the path of the test.

The former is best illustrated this way:

	unittest {
		assert(sqrt(16) == 4);
		assert(sqrt(4) == 2);
		assert(sqrt(1) == 1);
	}

Knowing which assertion fails in the given test might help to solve the problem, so it's legitimate in my view.

And here's an illustration of a second test modelling a sequence of events. If one of these test fails, all others will fail too because they depend on each other:

	unittest {
		S s = new S;
		s.giveMeAName();
		assert(s.fileName);
		s.value = 1;
		assert(s.value == 1);
		assert(s.needsToBeSaved);
		s.save();
		assert(!s.needsToBeSaved);
		s.value = 2;
		assert(s.value == 2);
		assert(s.needsToBeSaved);
		s.revert();
		assert(s.value == 1);
		assert(!s.needsToBeSaved);
		...
	}

So here you only need to know about the first error, and the rest is garbage.

It also highlights another problem about the current behaviour: what happens if one of the functions called in this test throws an exception? My guess is that it would abort unit tests for the whole module... I don't think anyone wants that.

-- 
Michel Fortin
michel.fortin at michelf.com
http://michelf.com/