[Issue 13454] Unit tests should be compiled in a module, where they are declared

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=13454

--- Comment #11 from Jonathan M Davis <issues.dlang at jmdavisProg.com> ---
It would certainly be nice to have a way to indicate that a unittest inside a
template is not actually part of the template, since it would allow you to put
the test for a function right after, which you can't do right now, which is
definitely ugly and harder to maintain, but I don't know how reasonable that is
from the standpoint of how templates work and how their implementation works.

As for compiling unit tests while importing, if I understand what Walter is
saying, he's saying that if you have the unit tests in the template, and
they're compiled into each instantiation when -unittest is used, then every
instantiation gets tested even if it's with template arguments that the
template's author never thought of, and that if you didn't want to test the
template with every instantiation, why did you put in the template in the first
place?

Now, that being said, while I can understand that point of view, as a user, I
don't think that I'd ever want a template that I imported to include unit tests
when I import it. I'd want the author to have tested it appropriately, and I
don't want to have the extra overhead in my app (even if it's just when unit
testing) of having those tests from a 3rd party compiled and run in my app.
Sure, it might catch some bugs, but just as likely, the tests weren't written
in a robust enough manner to work with all template arguments that the template
actually works with. And even if the tests work perfectly, I just don't want
that overhead. That belongs in the unit test app of the author of that code -
not in mine.

And as the author of a template, there are cases where I'd like to able to put
a test inside of the template and have it tested with all of the instantations
that I have in my unit test app and be able to have the unittest blocks right
next to what they're testing, and I don't want that code ending up in a user's
app. I don't want to worry about whether I wrote the tests robustly enough to
work with _all_ template arguments. Even if the tests are reasonably well
written, implicit conversions and the like can easily make it so that you end
up with a compilation error in the tests with a stray type that you didn't test
with and yet still works with the template just fine.

So, having unittest blocks be compiled in when compiling anything other than
the module that that template is in seems highly negative to me. I'd much
rather argue that it's up to the author to make sure that they test it with a
large enough range of template arguments rather than have those tests end up in
a user's code. And I'd hate to have to move unittest blocks out of a template
just so that they don't end up in user code.

Honestly, I think that the situation with templates and unit tests sucks. Once
in a while, I'm able to put unit tests directly in the template, because
they'll work with most or all template instantations, but if that's going to
end up in user code, then I'm _never_ putting them in there, which sucks. And
it sucks that in most cases, I have to put the tests outside of the template,
because I need to write tests which are for specific types, and you can't put
those inside of the template unless you do something like version each of them
so that they get compiled in with the instantiation that they go with. So, in
most cases, unit tests for templated types end up outside the template itself,
which harms maintainability. And given that I now know that tests within the
template end up in user code (something that I would never have expected), I
don't _ever_ intend to put unit tests inside of a templated type. That behavior
is completely undesirable to me. So, the current behavior is just plain bad for
maintainability IMHO.

I would love it if you could mark tests within a template as existing outside
the template - that would definitely improve maintainability of templated types
- and I think that for unit tests in templates to be worth using, they really
need to only be compiled in with the module that they're in. So, while I can
understand Walter's position from a flexibility and correctness standpoint,
from a usability and maintainability standpoint, I completely disagree with it.

--