[phobos] Silent failure of std.container unittests

[Leandro Lucarella]

Michel Fortin, el 15 de julio a las 18:32 me escribiste:
> 
> Le 2010-07-15 à 18:00, Walter Bright a écrit :
> 
> >> 4. DESIRED: assert is NOT hacked, any failing assert ends the current unittest, the failure message is printed, execution continues with the next unittest, program ends with error code if at least one assert failed, everybody's happy.
> > 
> > Everybody's happy except the guys like me who want to run all the unit tests in one go.
> 
> I think this is the root of the misunderstanding. Either you consider each assert grouped in a unittest blocks to be a unit test in itself; or you consider each unittest block to be a test with multiple possible points of failure across the path of the test.
> 
> The former is best illustrated this way:
> 
>       unittest {
>               assert(sqrt(16) == 4);
>               assert(sqrt(4) == 2);
>               assert(sqrt(1) == 1);
>       }
> 
> Knowing which assertion fails in the given test might help to solve the problem, so it's legitimate in my view.
> 
> And here's an illustration of a second test modelling a sequence of events. If one of these test fails, all others will fail too because they depend on each other:
> 
>       unittest {
>               S s = new S;
>               s.giveMeAName();
>               assert(s.fileName);
>               s.value = 1;
>               assert(s.value == 1);
>               assert(s.needsToBeSaved);
>               s.save();
>               assert(!s.needsToBeSaved);
>               s.value = 2;
>               assert(s.value == 2);
>               assert(s.needsToBeSaved);
>               s.revert();
>               assert(s.value == 1);
>               assert(!s.needsToBeSaved);
>               ...
>       }
> 
> So here you only need to know about the first error, and the rest is garbage.
> 
> It also highlights another problem about the current behaviour: what
> happens if one of the functions called in this test throws an
> exception? My guess is that it would abort unit tests for the whole
> module... I don't think anyone wants that.

This is why unit testing frameworks usually have 2 type of checks, one
that exists the current unit test if it fails and one that doesn't.
I think all we need is option 4) and something like:
import std.test;

        unittest {
                check(sqrt(16) == 4);
                check(sqrt(4) == 2);
                check(sqrt(1) == 1);
        }

std.test could have other convenience functions, like something like:
checkOp!"=="(sqrt(16), 4);
assertOp!">"(pow(x, y), 0);

Which prints (when failed because sqrt(16) returned 5 and pow(x, y)
returned -2) something like:
Check failed: 5 == 4
Assert failed: -2 > 0

It's a little uglier to write but provides more useful information.


-- 
Leandro Lucarella (AKA luca)                     http://llucax.com.ar/
----------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------
Todo caminante merece una palangana con agua caliente. Del reposos de
sus pies depende la longitud del camino.
        -- Ricardo Vaporeso. Valle de la Muerte, Junio de 1917.