unittest "name" {}

[Steven Schveighoffer]

On 2/11/23 4:10 PM, Bradley Chatha wrote:
> On Saturday, 11 February 2023 at 20:05:38 UTC, max haughton wrote:
>> That being said I am in favour of the language dictating that a string 
>> in a UDA on a unittest has special meaning (i.e. is a name or 
>> description)
> 
> Wouldn't that only really be useful if the compiler had better support 
> for named unittests, e.g. `-test-filer=yada`, or `-test-output=pretty 
> (similar to silly or something like that)`?
> 
> Or am I missing the point entirely here :D

All this is possible. Let me talk a little bit about how this all works.

The compiler takes all unittest blocks compiled in a given module, and 
creates individual functions from all of them.

Then, the compiler creates *one* master unittest function for each 
module, and has that function just call all the individual functions. 
This is the function pointer that is stored in the ModuleInfo.

That's all that is done from the compiler side. None of this is defined 
explicitly in the spec or language, so we have free reign to do this 
however we want.

Now, on the library side, all it does is search all moduleinfos and find 
any that have unittests. Then it runs those unittests using a function 
in druntime (or you can register a custom runner if you want). All that 
library function has accessible in the moduleinfo is the function 
pointer to run all unittests for that module. The unittest runner 
returns a structure that indicates how many tests were run (modules), 
and how many passed. We could easily modify that structure to say how 
many individual unittests were run (we can do whatever we want in the 
test running function, it has no explicit specification), and have the 
test runner collect which tests failed based either on file/line number 
by default, or use a UDA string if it's provided.

We could easily modify the moduleinfo to contain an array of unittest 
functions, and run that instead. Each function might even be a tuple of 
unittest name and function pointer. Then we have the capability to 
specify which unittests are run.

You can already do this at the module level if desired without any 
changes (e.g. --runUnittestModule=foo.bar can be acted on properly).

All of it is possible. We are very much unrestricted on how unittests 
actually run, since the only requirement is that the library run them, 
and report what happened.

The only thing about this is, it's all doable *already* with the test 
frameworks in code.dlang.org. It takes a bit more effort, but it's all 
available there. You aren't using the ModuleInfo system, but rather 
compile-time introspection to find unittests. But so what?

So really, it's a matter of how much we want to complicate the existing 
system to achieve something that is already somewhat achievable. That is 
the question we have to answer.

-Steve