Should unittests run as logical part of compilation?

[Idan Arye]

On Saturday, 25 January 2014 at 22:55:33 UTC, Andrei Alexandrescu 
wrote:
> There's this simple realization that unittests could (should?) 
> be considered an intrinsic part of the build process. In order 
> for an executable to be worth running, it should pass the 
> regular semantic checks and also the unittests, which in a 
> sense are extended semantic checks that fall outside the 
> traditional charter of the compiler.
>
> In that view, asserts inside unittests should fail with the 
> same message format as regular compilation errors, i.e.
>
> ./modulename.d(103): Unittest failed: user-defined message
>
> Stack traces and other artifacts printed by failing asserts 
> should come after, indented. An IDE user should run unittest 
> with the usual "build" command and be able to click on the 
> unittest failures just the same as build errors.
>
> In particular, this view of unittests declares our current 
> stance on running unittests ("run unittests just before 
> main()") as meaningless. Indeed that has bothered me for quite 
> a while - unittests are part of the build/acceptance, not part 
> of every run. To wit, this is a growing idiom in D programs:
>
> version(unittest) void main() {}
> else void main()
> {
>     ...
> }
>
> What do you think? Logistically it shouldn't be too hard to 
> arrange things to cater to this approach.
>
>
> Andrei

I don't think unit tests should be run on *each* build - if I'm 
fidgeting with a specific piece of my code, and building&running 
it after each change, I don't want to waste time running the 
unittests of the entire project.

I do agree that unittests shouldn't be part of each run though. I 
think the `-unittest` flag should ignore the `main` function 
altogether, creating a unittests only executable, and the build 
system will be responsible for running it.

How about keeping the old behavior of `-unittest` for backward 
compatibility, and having a new syntax - `-unittest=filename`, 
where "filename" is the name of the unittests executable. That 
way both the unittest and the regular executables will be created 
at the same time - which will speed things up, as most of the 
compilation work can be shared between those two builds - and 
have different file names, so the build system - which will make 
configuring the build system more straightforward.

If we make the unittest a separate executable, there won't be a 
`main` function waiting for command line arguments, so we can use 
the command line arguments for the unittests. We can have flags 
for only running tests of specific packages, or for running the 
tests under certain restrictions etc.