Named unittests

[H. S. Teoh]

On Sat, May 18, 2019 at 02:50:52AM +0000, Adam D. Ruppe via Digitalmars-d wrote:
> On Friday, 17 May 2019 at 21:56:45 UTC, Andrei Alexandrescu wrote:
> > Again: the entire unittest workflow should be designed, handled, and
> > marketed as an extension of the semantic checking process. The fact
> > that it's done after code generation is a minor detail.
> 
> I like this a lot. And I think I have a plan to make it work nicely
> with only one change to the compiler (when -unittest is used, the
> compiler automatically runs the test program before exiting
> successfully) and a few small changes to druntime (the tests are run
> only when given a particular command line flag).

Dmd already has -run, so it's not too much of a stretch to change the
behaviour of -unittest into the equivalent of today's:

	dmd -unittest -run blah.d

Furthermore, main() shouldn't run when unittests are run, otherwise this
wouldn't make sense (compiling the program shouldn't execute main() as a
side-effect).  Perhaps some kind of hidden switch to suppress the
emission of main() so that the compiler can run the unittests after
compilation without also triggering main().  Perhaps tentatively we can
call it --DRT-run-unittests, which will start the unittest runner and
skip over calling _Dmain.


> D's built in unittests are easy enough you might as well write them,
> it is little effort.

D's built-in unittests are currently *the* reason I write any unittests
for my programs at all.  I wouldn't want to go without them.

True, one of my recent projects needed external testing, so I've written
a simplistic external test driver for this purpose (it's driven by a
bunch of files in a subdir called test/, and figures out what to do
based on what input files exist, so creating a test is a matter of
writing an input file and one or more output files containing expected
output). But for targeted functionality I still prefer the built-in
unittests, even in addition to the external test suite.


> This would make running them so seamless you don't even realize it is
> happening too. (heck I kinda want -unittest to become a default, so
> dmd runs it unless you specifically tell it not to. but that's another
> conversation).
[...]

I like the idea of -unittest by default.

One useful pattern that we could consider, that I've developed over
time, is to have the compiler compile *two* executables for the same
code, one with unittests (with no main()) and the other without (and
with main()). My build script runs both in parallel, and automatically
executes the unittest executable as part of the build. If a unittest
fails, the build aborts with an error. Otherwise, it deletes the
unittest executable, leaving the "real" one ready to run.

Of course, currently this involves running two instances of the
compiler, one with -unittest and one without; but if there was some way
to unify them, and perhaps make running the unittests automatic, that
would be *really* nice.

Then we could make this the default behaviour, and have a -no-unittest
to suppress it and revert to today's behaviour.


T

-- 
If it tastes good, it's probably bad for you.