Extending unittests [proposal] [Proof Of Concept]

Johannes Pfau nospam at example.com
Fri Sep 21 08:35:38 PDT 2012


Am Fri, 21 Sep 2012 16:37:37 +0200
schrieb Jens Mueller <jens.k.mueller at gmx.de>:

> Jacob Carlborg wrote:
> > On 2012-09-21 14:19, Jens Mueller wrote:
> > 
> > >Why do you need filename and line information of a unittest. If a
> > >unittest fails you'll get the relevant information. Why do you

With the recent name mangling change it's possible to get the unittest
line if a test fails, but only if you have working backtraces. That
might not be true for other compilers / non x86 architectures.

To get the filename you have to demangle the unittest function name
(IIRC core.demangle can't even demangle that name right now) and this
only gives you the module name (which you could also get using
moduleinfo though)

It's also useful for disabled tests, so you can actually look them up.


> > >want the information when a unittest succeeded? I only care about
> > >failed unittests. A count of the number of executed unittests and
> > >total number is enough, I think.

The posted example shows everything that can be done, even if it might
not make sense. However printing successful tests also has a use case:

1: It shows the progress of unit testing. (Not so important)
2: If code crashes and doesn't produce a backtrace, you still now which
test crashed as the file name and line number are printed before
running the test. (might sound unprobable. But try porting gdc to a new
architecture. I don't want to waste time anymore commenting out
unit tests to find the failing one in a file with dozens of tests and
an ARM machine that takes ages to run the tests)

Another use case is printing all unittests in a library. Or a gui app
displaying all unittests, allowing to only run single unittests, etc.

Of course names are better than filename+line. But names need a change
in the language and filename+line are an useful identifier as long as we
don't have names.

> > 
> > But others might care about other things. I doesn't hurt if the
> > information is available. There might be use cases when one would
> > want to display all tests regardless of if they failed or not.
> 
> If there are use cases I agree. I do not know one.
> The question whether there are *tools* that report in case of success
> is easier to verify. Do you know any tool that does reporting in case
> success? I think gtest does not do it. I'm not sure about JUnit.

I don't know those tools, but I guess they have some sort of progress
indicator?

But I remember some .NET unit test GUIs that showed a green button for
successful tests. But it's been years since I've done anything in .NET.

> But of course if a unittest has additional information and that is
> already implemented or easy to implement fine with me. My point is
> more that for the common cases you do not need this. Maybe in most.
> Maybe in all.

You usually don't have to print sucessful tests (although sometimes I
wasn't sure if tests actually run), but as you can't know at compile
time which tests fail you either have this information for all tests or
for none.

The main reason _I_ want this is for gdc: We currently don't run the
unit tests on gdc at all. I know they won't pass on ARM. But the unit
tests error out on the first failing test. Often that error is a
difficult to fix backend bug, and lots of simpler library bugs are
hidden because the other tests aren't executed.

I'm actually kinda surprised the feedback on this is rather negative. I
thought running unit tests individually and printing line/file/name was
requested quite often?


More information about the Digitalmars-d mailing list