How can one reliably run unittests

[deadalnix]

D's unittest block is a great idea. However, I find that in 
practice, I fail to make good use of them and have been 
neglecting them. Sometime by using an external test suite 
instead, sometime neglecting testing altogether. How come?

Well, the main reason is that, as far as I can tell, it is near 
damn impossible to run them at scale in any sensible way. Let me 
explain.

Ideally, one would want a module to be a unit. Declaration, 
implementation, but also tests are part of the unit. During the 
dev cycle, it is desired to be able to runt he test on modules to 
verify that everything works as advertised. While doing so is 
easy on a pet project,a s the projects grows, contains several 
libraries and executable, doing this become very challenging.

It is not really possible to build a monster executable that 
contains everything. First, this would kill build times, but, 
with several main around it won't link. So let's split into 
component.

Some of these component are libraries, some are executable. 
Adding a main for libraries is required, or it won't link, but 
adding one to executable is going to cause a link error. This in 
itself is a major main in the ass, because that means there is no 
one consistent way to unittest a module without knowing if that 
module has a main. In addition, everything needs to be built 
twice now, which is really undesirable.

Maybe one could rdmd each modules to runt he tests? That seems 
like the best approach to me, considering one doesn't want to 
actually produce an artifact for the tests, simply run them. this 
also has the main/no main problem, but in addition, it often 
fails with incomprehensible linker errors. For some reason, rdmd 
is not able to include all dependencies consistently, but, even 
better, it doesn't take the same standard flags as other linkers 
do, so it is not possible to reuse existing build infrastructure 
to feed all the correct flags to rdmd.

This may seems like it wouldn't be that big of a deal if you 
manage all your flags by yourself, but very quickly turns into a 
nightmare once you have to use 3rd party libraries that ship with 
their own set of flags.

At this point, I would just wish that one could rdmd --unittest 
modulename.d and just pass it a couple of -I, -L and -l flags and 
have all of it work. I have no idea how to achieve that.

This kind of orthogonality is important as a project scale. One 
cannot just doctor all the unitests call to all the modules. You 
need to be able to tell your build system something akin to "hey, 
every time you encounter a D module, please implicitly add this 
unitest target. Thanks." This cannot be made to work if running 
unit tests is done in a way that depends on the content of the 
module.

I don't think my case is isolated, every single substantial D 
project I encountered has some kind of test suite or something 
similar instead of relying on the unitests within modules. This 
is frankly a failure, and it is 100% due to poor UX as the 
language feature itself is great.

If someone has some bandwidth, it is IMO a high leverage task 
that solve real problem for real people and 90% of the work is 
already there. The feature is there, the runtime support &al is 
there. just provide a consistent UX so its use can be integrated 
properly in larger systems.

Thanks in advance.