How can one reliably run unittests

[Alexandru Ermicioi]

On Tuesday, 24 August 2021 at 12:39:47 UTC, Atila Neves wrote:
> Interesting.
>
> My ideal scenario wouldn't involve linkers at all - I don't 
> want to spend time waiting for my code to link, especially 
> since, the way I write code means that unit tests never need 
> optimising or native code generated. I'd go as far as to say 
> that if one's unit tests *do* need any of that, they need to 
> rethink their mini-integra... err, unit tests. But I digress.
>
> The way I'd like it to work personally would be to CTFE unit 
> tests in a module and have them run whenever I save the file. 
> No linker, no main function, just that.
>
> Back to the issue at hand. The problem with wanting to pass 
> flags to the rdmd is that unless the project is trivial it'll 
> have dub dependencies, so doing that manually is a no-no. This 
> is the same thing that happens all the time in 
> compile-as-you-type plugins, and why I maintain two separate 
> emacs packages to make that done automatically (one for 
> CMake/C++ and one for dub/D). Which means asking dub. Which 
> means a build system.
>
> Which means you just gave me an idea for an awesome reggae rule 
> to generate a library with all of the dependencies of a module, 
> a main function to run the tests, and then voilà. It'd have to 
> do this for each module, and the first time said 
> all-dependencies library is built it'd take time, but after 
> that it'd work.
>
> You'd still have to wait for the linker though, and if for some 
> reason you're not using lld (or worse, you're on Windows), the 
> wait will be looooong.

Imho, there should be a well known api for fetching unit tests at 
compile time and at runtime, then external or builtin unit test 
runner could use the api to properly select which unit tests to 
run. Second thing that compile time api should offer is the 
ability to force compiler ignore unit tests from modules or 
packages or annotations on unit test itself (it is also a 
nightmare to compile unit tests from a third party source library 
that takes ages to compile due to template bloat), or in general 
provide some kind of hooks into unittest config and running, 
similar to how junit does in java.

Best regards,
Alexandru.