unittests are really part of the build, not a special run
Idan Arye via Digitalmars-d
digitalmars-d at puremagic.com
Tue Mar 31 01:56:42 PDT 2015
On Monday, 30 March 2015 at 22:20:08 UTC, Andrei Alexandrescu
> This is a tooling issue.
> In brief: I'd like to transition to a model in which
> unittesting is organically part of the build. After all, you
> wouldn't want to deploy an application that's failing its
> Detail: Consider running a build vs. a unittest in one of the
> supported editors/IDEs (emacs, vim, Code:Blocks, Visual D,
> Xamarin...). During a build, an error will come in a standard
> format, e.g.
> std/array.d(39): Error: undefined identifier xyz
> This format is recognized by the editor and allows the user to
> click on it and go straight to the offending line etc.
> In contrast, a failing unittest has a completely different
> format. In fact, it's a format that's worse than useless
> because it confuses the editor:
> core.exception.AssertError at std/array.d(39): xyz
> Where I want us to be is a place where unittests are considered
> a part of the build; it should be trivial to set things up such
> that unittesting is virtually indistinguishable from
> compilation and linking.
There is no point in running unittests before `main` in the same
executable, but that doesn't mean the build is the right place
for running unittests. The IDE/build-system should be the one
that handles running the tests(both unit and integration). The
compiler should give the IDE/build-system enough tools to do it
properly - not to do it for them. Running UTs as part of the
build blocks some options, like building a UT executable and
running it in another environment where building is not possible.
Ideally, I would like to see the compiler, when ordered to do a
unittest build, create an executable that only runs unittests and
ignores `main`. In matter of fact, what it does is run a special
"ut-main" entry point function declared in Phobos(or in the
runtime - whichever makes more sense) that runs all the
unittests, and can possibly catch `AssertError`s and display them
in proper format.
Since we no longer run `main`, the command line arguments can go
to the ut-main function. This doesn't change anything with the
built-in ut-main since it just ignores them, but if that special
entry point can be overrideable via a compiler flag, and
IDE/build-system can supply it's own, more complex ut-main that
can make use of the command line arguments. That special ut-main
can communicate with the IDE/build-system to provide a better UX
for unittest running - for example an IDE could do a graphical
display of the unittests run, when it's custom ut-main is
responsible for providing it info about the ut-run progress.
More information about the Digitalmars-d