Why think unit tests should be in their own source code hierarchy instead of side-by-side

[Atila Neves]

On Thursday, 22 March 2018 at 16:30:37 UTC, H. S. Teoh wrote:
> On Thu, Mar 22, 2018 at 10:59:56AM +0000, Atila Neves via 
> Digitalmars-d-announce wrote:
>> Blog post:
>> 
>> https://atilanevesoncode.wordpress.com/
> [...]
>
> I realize this is your opinion, but I disagree with them 
> because:

Disagreeing is more than fine. :)

> 1) I've found that having unittests built into the language is 
> a big win, *especially* because I can write tests next to the 
> code being tested.

I'm confused. I don't know what I wrote that made you think I 
believe otherwise. I wrote "fantastically successful for the 
language" and that "Let me start by saying that some tests should 
go along with the production code".

> being tested, was a big slowdown for me.  I have to stop to 
> think about which subdirectory under test/ I should put the 
> relevant file(s), or if there's already a test there I have to 
> stop to think about which filename I saved it under, etc..  
> It's just yet another mental hurdle to jump over while my 
> already-busy brain is thinking about the code itself.

Then I'd recommend:

1) To write the tests inline when the mental burden is high
2) Move them afterwards when one's brain can think of where to 
place them.

I think that mirroring the production source tree is probably the 
way to go most of the time, i.e. test/foo/bar/baz.d for 
src/foo/bar/baz.d.

> 2) Compilation times: perhaps if your unittests are too heavy, 
> compilation times could become a problem,

Maybe I wasn't clear in what I wrote: I'm not saying that I 
notice the increase in compilation times of the tests themselves. 
I probably haven't but I'd have to measure to know for sure. I'm 
saying that if you change anything in a D module, it must be 
assumed that any other module that imports the module you just 
edited, even if transitively, must be recompiled. So if I edit a 
test in D, under normal circumstances, I _have_ to recompile 
several files. It's not the unittest itself that is heavy, it's 
recompiling everyone who depends on the module that said unittest 
happens to find itself in.

> but IME, they haven't been.

IME most other people find it bizarre I get angry at 2s 
incremental rebuild times. Anything over 100ms is an eternity.

> Plus, my opinion is that when you're compiling the code, you 
> *should* be running unittests anyway (otherwise regressions 
> inevitably slip in),

Yes.

> so you're going to have to pay for the time taken to compile 
> them regardless.

Yes.

> In that sense, it's actually better to have them in the same 
> file so that the
> compiler doesn't have to open up another file and allocate 
> resources for handling another module.

Noooooooooo.

> As for the dub-specific problems introduced by 
> version(unittest): IMO that's a flaw in dub.

It's not dub-specific at all. It's same problem that you 
reference in:

> I remember that in Phobos we used to merge PRs containing code
> that compiles fine with -unittest, but in real-world code 
> doesn't compile because it has stuff outside unittests that 
> depend on imports/declarations inside version(unittest).

I used dub as an example. Anyone else would have the same 
problems if they hand-wrote Makefiles using git submodules as 
dependencies.

Atila