How can one reliably run unittests

[jfondren]

On Thursday, 9 September 2021 at 00:13:00 UTC, James Blachly 
wrote:
> Let's suppose I maintain a library L (written in D; has unit 
> tests). Suppose further I have client program P, which uses L.
>
> As far as I can tell, if P's dubfile includes library L, and 
> the user execeutes `dub test` then the unit tests from the 
> library L as well as program P are built and executed.
>
> We had to wrap our library's unittests in 
> `debug(library_unittest)` to escape this behavior.
>
> Can you provide some clarity because it sounds like you are 
> saying dub will build P's unit tests, but not library L's? Not 
> at all my experience, unless it was updated recently.

Either one of the stated behaviors can be the case.

Consider these pair of libraries:

```
$ grep -R .
ell/dub.sdl:name "ell"
ell/dub.sdl:description "A minimal D application."
ell/dub.sdl:authors "Julian Fondren"
ell/dub.sdl:copyright "Copyright © 2021, Julian Fondren"
ell/dub.sdl:license "MIT"
ell/source/ell.d:module ell;
ell/source/ell.d:public const int x = 5;
ell/source/ell.d:unittest {
ell/source/ell.d:    assert(x != 5);
ell/source/ell.d:}
pea/dub.sdl:name "pea"
pea/dub.sdl:description "A minimal D application."
pea/dub.sdl:authors "Julian Fondren"
pea/dub.sdl:copyright "Copyright © 2021, Julian Fondren"
pea/dub.sdl:license "MIT"
pea/dub.sdl:dependency "ell" version="*"
pea/source/pea.d:module pea;
pea/source/pea.d:import ell : x;
pea/source/pea.d:public const int n = 5;
pea/source/pea.d:unittest {
pea/source/pea.d:    assert(n == x);
pea/source/pea.d:}

```

ell has tests that always fail. pea has tests that always 
succeed. pea depends on ell.

```
$ dub add-local ./ell
Registered package: ell (version: ~master)
$ cd pea
pea$ dub -q test
1 modules passed unittests
```

In this configuration, that ell fails its unittests doesn't 
matter at all to pea. The library and its value `x` are still 
used in pea.

And ell definitely fails its unittests:

```
pea$ cd ../ell; dub -q test
core.exception.AssertError at source/ell.d(6): unittest failure
...
... a lot of output
...
1/1 modules FAILED unittests
Program exited with code 1
```

But let's add a single line to ell's dub configuration...

```
ell$ echo 'targetType "sourceLibrary"' >> dub.sdl
ell$ cat dub.sdl
name "ell"
description "A minimal D application."
authors "Julian Fondren"
copyright "Copyright © 2021, Julian Fondren"
license "MIT"
targetType "sourceLibrary"
```

...and we see the behavior that you noted with P and L:

```
ell$ cd ../pea
pea$ dub -q test
core.exception.AssertError at ../ell/source/ell.d(6): unittest 
failure
...
... a lot of output
...
1/2 modules FAILED unittests
```

IMO, D's #1 problem is that dub's documentation is 1/10000 the 
length it needs to be. (Meanwhile other problems with dub only 
show up in the three digits, like D problem #451: a dub !# script 
does a bunch of git and repository stuff with undesirable 
slowdowns.)