Why think unit tests should be in their own source code hierarchy instead of side-by-side

[H. S. Teoh]

On Fri, Mar 23, 2018 at 03:56:03PM -0400, Steven Schveighoffer via Digitalmars-d-announce wrote:
[..]
> Note that a frequent complaint of std.datetime (at least when it was
> one module) is that the file was too big. While it does hold a lot of
> functionality, the majority of the file size is unittests. This means
> that it can be hard to surf the file for functionality.

Actually, I didn't have a problem with std.datetime being a huge file.
What I *did* have a problem with was that it took too much memory to
compile and the unittests took too long to run.  And also that it
consisted of a bunch of somewhat loosely-coupled chunks that could be
more meaningfully encapsulated as submodules.


> But on the flip side, there aren't a lot of datetime bugs!

Exactly.


> I personally believe that there should be unit tests for every
> function, and be right next to the function. I don't want to go on a
> search for such things, or have to rely on manual documentation to
> know what is testing what. It would be nice to have your editor hide
> the unit tests unless you want to work on them.

It's just a matter of configuring folding in an editor that supports it.
Or dump your editor if it doesn't support folding, 'cos it sux and you
deserve better. :-D  I mean, this is 2018, not 1995, we shouldn't have
to be stuck with the handicap of navigating by scrollbar and paging
up/down anymore.


> I've worked on a project where the testing was separated from the
> code, and it was a liability IMO. Things would get missed and not
> tested properly.
[...]

Yep.  As I mentioned elsewhere, recently I've had to resort to external
testing for one of my projects, and I'm still working on that right now.
And already, I'm seeing a liability: rather than quickly locating a
unittest immediately following a particular function, now I have to
remember "oh which subdirectory was it that the tests were put in? and
which file was it that a particular test of this function was done?".
It's an additional mental burden to have to keep doing the mapping
between current source location <-> test code location (even if it's a
1-to-1 mapping), and a physical burden to have to continually open
external files (and typing a potentially long path for them) rather than
just "bookmark, jump to end of function, navigate unittest blocks" in
the same file.

Fortunately, this problem is somewhat alleviated by having cumulative
code coverage (see other thread), which can quickly identify what hasn't
been tested yet.  Currently, with the help of coverage, I've managed to
get 100% coverage for 4 of my modules, and 90%+ for another 5 modules,
and working on the remainder right now.  It's an uphill battle at times,
but already I've been reaping the benefits: buggy corner cases that I
missed were discovered, fixed, and now have a corresponding test case /
unittest where possible.


T

-- 
I've been around long enough to have seen an endless parade of magic new techniques du jour, most of which purport to remove the necessity of thought about your programming problem.  In the end they wind up contributing one or two pieces to the collective wisdom, and fade away in the rearview mirror. -- Walter Bright