This is why I don't use D.

[Jonathan M Davis]

On Saturday, September 8, 2018 7:44:09 AM MDT Nicholas Wilson via 
Digitalmars-d wrote:
> On Friday, 7 September 2018 at 19:15:21 UTC, Jonathan M Davis
>
> wrote:
> > Honestly, I wouldn't rely on anything beyond dub build working
> > in a consistent manner across projects. As far as I can tell,
> > you can't actually do anything properly custom with dub test,
> > and I'm inclined to think that how it approaches things is
> > outright wrong.
>
> For Dcompute I define a unittest configuration that has a main
> that runs tests, and the (default) library configuration excludes
> source files in the test directory. It seems to work fine.
> Admittedly it is a little different to you usual package, and
> there are no test specific symbols and no unit tests, and I
> haven't tested anything that has dcompute as a dependancy.
>
> Are you suggesting this will break users of Dcompute that run dub
> test in their code? If so that truly does suck, because regular
> unittests will not cut the mustard.

Okay. It's been several months since I dug into this, so hopefully I have
the details straight.

Unless no one using dcompute uses dub test on their own projects, you
probably don't have the worst part of this problem, because if you did,
they'd be complaining about linker errors.

Honestly, unittest blocks in general are somewhat broken, but what really
doesn't work correctly is code that's versioned for unit tests. e.g. if
you've declared some types or functions to use specifically in your unit
tests and which have no business being in the actual library (e.g. dxml has
dxml/internal.d which includes stuff specifically to help the unit tests and
is not part of the public API). The core problem is caused by the language,
but the way dub handles things negates the workarounds for the problem.

When you import a module that includes unittest blocks, and you're compiling
your module with -unittest, the version identifier unittest is defined when
importing the module, meaning that all unittest blocks are processed, and
everything inside a version(unittest) block is considered to be part of the
module. With regards to the unittest blocks themselves, this is nothing but
negative. It means that they have to compile regardless of whether
everything they use is available to the module importing them, and it's a
pointless waste of compiler resources. This is even worse when a unittest
block is inside a template, because than anyone using that template then
also gets the unittest blocks compiled into their code (once for each
instantation of the template), meaning that the unittest blocks in the
libraries you use can actually be compiled into your program and get run
when you run your own unit tests. IMHO, when importing a module, unittest
blocks should be treated as version(none) rather than version(unittest), but
that's not currently the case.

The less straightforward problem is the helpers that are compiled with
version(unittest). We can't treat everything that's version(unittest) as
version(none) when importing, because that would make it impossible to share
unit test helpers between modules. The cleanest way to deal with those is
therefore to put them in a separate module that gets imported locally by the
unittest blocks. That way, you don't accidentally mark anything in the
module with the unittest blocks as version(unittest), and it allows you to
hide the symbols from any modules that import your module. However, because
the unittest blocks themselves are still processed when importing, it
doesn't actually work to hide the symbols being imported like it should. The
compiler still processes them when it imports the module. This causes
problems when the modules being imported weren't actually compiled into the
library that you're linking against (since normally, libraries aren't
compiled with -unittest) - hence the linker errors. If you use unit test
helpers that are version(unittest) in a library, you risk linker errors in
any project that imports that library. Not marking them with
version(unittest) can fix that problem, but it then pollutes projects using
the library.

While it's certainly ugly, the easiest way to solve the problem at present
is to version all of your unittest blocks with a version identifier specific
to your project. That way, any projects depending on your library won't
actually end up compiling the unittest blocks, because while the unittest
version identifier will have been declared, the version identifier specific
to your project will not have been. There was actually a PR for Phobos a few
months back that tried to do this to all of Phobos to fix this problem with
Phobos, but it was rejected on the grounds that it was hideous and that we
should solve it in the language - which I agree with, but in the interim, we
still have to deal with the problem - hence why dxml has the dxmlTests
version identifier. Without it, any projects using dxml would get linker
errors when running their own tests.

Where dub makes this whole situation worse is that for some reason, the
build configuration for your library that's used when you run dub test is
also used by anything that pulls in your library via dub. I had made it so
that dub test declared dxmlTests so that dub test would work properly for
dxml but with the idea that then any projects depending on dxml would not
declare dxmlTests. However, because dub pulls in the dependencies' unit test
configurations instead of just using the non-unit test one like it should,
that results in any projects that depend on dxml having dxmlTests declared,
so the problem isn't fixed, and all of those annoying version(dxmlTests)
blocks were a waste of time.

To workaround this stupidity of dub, I was forced to create a separate set
of buildTypes for running the unit tests in debug, release, or with code
coverage. Those buildTypes must be used, or there are no unit tests, meaning
that dub test goes really fast and does nothing useful whatsoever. And
because of this whole issue, I avoid dub test like the plague.

So, dub needs to fix what it's doing (I can't remember what I reported at
the time - I really should make sure that I have all of the facts straight
and clearly explained in a bug report for dub), and once it is fixed, then
dub test can be used again by projects that need to do custom stuff with
their unit test configurations. But that still leaves the compiler problem,
and a DIP needs to written to fix that. IIRC, Jonathan Marler was trying to
fix it several months ago, but I think that he was mucking around with
version(unittest) in general, and he ran into all kinds of weird problems.
So, I don't know what it would really take to fix the problem in the
compiler, but I think that it's pretty clear that unittest blocks should be
utterly ignored when the module they're in is imported, and right now,
they're not.

- Jonathan M Davis