'live' testing style

[Jonathan M Davis]

On Tuesday 15 February 2011 00:46:25 spir wrote:
> On 02/15/2011 02:42 AM, Jonathan M Davis wrote:
> > On Monday, February 14, 2011 17:26:26 Don wrote:
> >> Jonathan M Davis wrote:
> >>> On Monday, February 14, 2011 12:49:11 Tomek Sowiński wrote:
> >>>> spir napisał:
> >>>>> * Why isn't testList a unittest block?
> >>>>> 
> >>>>> Using named funcs, I can switch on&  off specific test suites by
> >>>>> (un)commenting their call from the main and unique unittest block.
> >>>>> Else, either they all run, or none. During development, I only keep
> >>>>> active the test func(s) relative to the feature I'm currently working
> >>>>> on. Remedy: named unittests.
> >>>> 
> >>>> The interesting thing about named unit tests is that their names
> >>>> aren't interesting at all. They are usually dull and forced; testing
> >>>> filterFoo will be called testFilterFoo, etc. Their only purpose is to
> >>>> suppress running of unrelated tests.
> >>>> 
> >>>> 
> >>>> Now, there is a seemingly unrelated proposal to include every ddoc'ed
> >>>> unit test in the preceding declaration as an example. This is great
> >>>> because it implies ownership -- a unit test is 'owned' by the symbol
> >>>> above. Going further, it can also be named after its owner.
> >>>> 
> >>>> module ooh;
> >>>> 
> >>>> void foo();
> >>>> 
> >>>> unittest { test foo... }
> >>>> 
> >>>> Compiling with --unittest=ooh.foo runs this unittest only. Nested
> >>>> control as a bonus: compiling with --unittest=ooh runs only the tests
> >>>> in module ooh.
> >>>> 
> >>>> So there you go, named unit tests without naming.
> >>> 
> >>> The number one reason that I want named unit tests is stack traces.
> >>> unittest428 or whatever a unittest block gets named to is pretty
> >>> useless. So, while an assert within the unit test is fine, since it
> >>> gives a file and line number which is in the unittest block, exceptions
> >>> thrown from stuff called by the unit test become hard to track down,
> >>> since you don't know which unit test was being run.
> >> 
> >> Sounds as though a solution would be to have unittests automatically
> >> named as "_unittest" ~ __LINE__
> >> rather than an arbitrary integer.
> >> Wouldn't work in the case where there are two unittests on the same
> >> line: unittest { assert(true); } unittest { assert(true); }
> >> But we can deal with that.
> > 
> > I still think that having named unit tests is something that we should do
> > at some point (and it would be nicely backwards compatible), but naming
> > the unit test function after the line number that it's on would already
> > be a big improvement.
> > 
> > I have no idea how they're named/numbered now. I haven't been able to
> > discern any obvious pattern, and with a module like std.datetime, which
> > has lots of unittest blocks, even if it were clearly unittest1,
> > unittest2, unittest3, etc. it would still be quite hard to figure out
> > which unittest block was running when the exception threw, since you'd
> > have to count them all.
> > 
> > Using __LINE__ would be a big improvement - though unless we figure out
> > some way of advertising that, most people probably wouldn't have a clue,
> > and it wouldn't help them much. It would definitely be a positive change
> > though.
> > 
> > And since if someone is bizarre enough to put more than one unittest
> > block on one line, well then they suffer for what is almost certainly a
> > poor choice. But it would still narrow it down for them.
> 
> The only /meaningful/ choice for a programmer, in the general case, is
> programmer-chosen names. Don't you think? One can then even nicely use
> those names like eg "see example use in unittest
> ApplicationToArithmetics". If people are happy with naming unittests using
> ordinals (1 2 3...) or line numbers (in realease), well... let them free,
> after all it's their choice :-)

I'm looking for usable stack traces from unit test blocks. Ideally, that would 
be with named unit tests, but that's a language change which may not happen for 
a while, if ever. Having the unittest block functions named with __LINE__ being 
used to generate the numeric part of their name would at least give me the 
information that I need to use stack traces from exceptions that went through a 
unit test block. So, while named unit tests would be nice, the __LINE__ bit 
would be a big help.

- Jonathan M Davis