Detached unit tests

[Jonathan M Davis]

On Wednesday, May 1, 2024 8:26:21 AM MDT Steven Schveighoffer via dip.ideas 
wrote:
> On Wednesday, 1 May 2024 at 12:55:01 UTC, Timon Gehr wrote:
> > Looks useful, as usually you want to test the API. Maybe this
> > should have been the default behavior for unit tests with
> > `public` visibility.
>
> Hm... `public unittest` syntax?
>
> The problem I see with that syntax is that `public:` is way way
> more likely than `static:`.
>
> Perhaps you mean ddoc'd unittests? Maybe...
>
> I dislike comments affecting code generation (one of the reasons
> I think we should never have `__traits(getDocumentation,
> symbol)`), and also you may want to ddoc private functions for
> internal docs.
>
> I'm not attached to the `static unittest` syntax, it's just
> what's handy. A `core.attribute` UDA would work fine for me.

I would expect it to be problematic for public or private attributes which
weren't directly on the unittest blocks to affect unittest blocks (and not
having public or private elsewhere affect unittest blocks would arguably be
bad for language consistency). unittest blocks which aren't intended to be
for documentation normally use private symbols or imports from the module,
and we'd be forced to slap private on them all over the place once this
feature was enabled. We'd also have to mark more unittest blocks with
attributes that way, because typically, there are more unittest blocks which
are not intended to be part of the documentation than those that are.

Having it be automatic for ddoc-ed unittest blocks makes far more sense,
because those are clearly intended to be part of the documentation and thus
arguably shouldn't be using any private symbols or using any imports from
the module at large (though for better or worse, it's certainly being more
pedantic about it if we force that for all ddoc-ed unittest blocks rather
than giving programmers the choice). However, that doesn't help us put
undocumented unittest blocks next to what they're testing within templates
without having those tests duplicated across every instantiation, and it
would be very nice if we could fix that as part of this (though that would
require that the compiler deal with templates differently than it does now,
whereas for non-templated unittest blocks, it's simply a question of adding
additional checks).

So, I think that we're probably best off if we require that detached unit
tests be explicitly marked as such.

But I'm also not all that concerned about what that explicit marking looks
like. static makes a lot of sense to me given what that means in general,
but I'd also be totally fine with with some other attribute, including
something like @detached.

- Jonathan M Davis