Why think unit tests should be in their own source code hierarchy instead of side-by-side

[Jonathan M Davis]

On Thursday, March 22, 2018 09:30:37 H. S. Teoh via Digitalmars-d-announce 
wrote:
> 2) Compilation times: perhaps if your unittests are too heavy,
> compilation times could become a problem, but IME, they haven't been.

Personally, I agree with you, but Atila is one of those folks who gets
annoyed when tests take even a fraction of a second longer, so stuff that
many of us would not consider pain points at all tends to annoy him. I
prefer it when tests run fast, but if they take 1 second rather than 500
milliseconds, I don't consider it a big deal. Atila would.

So, given Atila's preferences, it makes sense to remove the tests from the
module if that speeds up compilation even a little, whereas personally, I'd
much rather have them next to the code they're testing so that I can see
what's being tested, and I don't have to go track down another file and edit
the tests there when I'm editing the code being tested. I'd prefer that that
not harm compilation time, but if it does, I'm generally going to put up
with it rather than move the tests elsewhere.

> 3) version(unittest): it's true that this can be a problem.  I remember
> that in Phobos we used to merge PRs containing code that compiles fine
> with -unittest, but in real-world code doesn't compile because it has
> stuff outside unittests that depend on imports/declarations inside
> version(unittest).  This is actually one of the reasons I was (and still
> am) a big supporter of local/scoped imports. It may be more convenient
> to just put global imports at the top of the module, but it just creates
> too many implicit dependencies from mostly-unrelated chunks of code,
> that I'm inclined actually to call global imports an anti-pattern.  In
> fact, I'd even go so far to say that version(unittest) in general is a
> bad idea.  It is better to factor out stuff inside version(unittest) to
> a different module dedicated to unittest-specific stuff, and have each
> unittest that needs it import that module.  This way you avoid polluting
> the non-unittest namespace with unittest-specific symbols.

I don't think that global imports are really an anti-pattern, though there
are advantages to local imports. The big problem with global imports is when
they're versioned, because then it's way too easy to screw them up. If
they're not versioned, then worst case, you're importing something which
wouldn't necessarily have to be imported if local imports were used.

> As for the dub-specific problems introduced by version(unittest): IMO
> that's a flaw in dub.  I should not need to contort my code just to
> accomodate some flaw in dub.  Having said that, though, I do agree that
> version(unittest) itself is a bad idea, so code written the way I
> recommend would not have this problem even given dub's flaws.

dub makes the problem worse, but it's inherent in how D currently works.
When I compile my project with -unittest, and I'm importing your library,
your version(unittest) code gets compiled in. I'm not sure exactly what
happens with the unittest blocks, since the code for them isn't generated
unless the module is explicitly compiled, but I think that it's the case
that all of the semantic analysis still gets done for them, in which case,
anything they import would need to be available, and that's a huge negative.
We really need a DIP to sort this out.

Now, unfortunately, dub makes the whole thing worse by compiling
dependencies with their unittest target when you build your project with its
unittest target, so you get all of the unit tests of all of your
dependencies regardless of what dmd does, but even if dub were not doing
that, we'd still have a problem with the language itself.

- Jonathan M Davis