Strategies for resolving cyclic dependencies in static ctors

[Graham St Jack]

On 25/03/11 06:09, Steven Schveighoffer wrote:
> On Thu, 24 Mar 2011 00:17:03 -0400, Graham St Jack 
> <Graham.StJack at internode.on.net> wrote:
>
>> Regarding unit tests - I have never been a fan of putting unit test 
>> code into the modules being tested because:
>> * Doing so introduces stacks of unnecessary imports, and bloats the 
>> module.
>
> As Jonathan says, version(unittest) works.  No need to bloat 
> unnecessarily.

Agreed. However, all the circularity problems pop up when you compile 
with -unittest.

>
>> * Executing the unittests happens during execution rather than during 
>> the build.
>
> Compile-time code execution is not a good idea for unit tests.  It is 
> always more secure and accurate to execute tests in the environment of 
> the application, not the compiler.

I didn't say during compilation - the build tool I use executes the test 
programs automatically.

>
> Besides, this is an implementation detail.  It is easily mitigated.  
> For example, phobos' unit tests can be run simply by doing:
>
> make -f posix.mak unittest
>
> and it builds + runs all unit tests.  This can be viewed as part of 
> the "Build process".

The problem I have with this is that executing the tests requires a 
"special" build and run which is optional. It is the optional part that 
is the key problem. In my last workplace, I set up a big test suite that 
was optional, and by the time we got around to running it, so many tests 
were broken that it was way too difficult to maintain. In my current 
workplace, the tests are executed as part of the build process, so you 
discover regressions ASAP.

>
>> All unittests (as in the keyword) seem to have going for them is to 
>> be an aid to documentation.
>>
>
> The huge benefit of D's unit tests are that the test is physically 
> close to the code it's testing.  This helps in debugging and managing 
> updates.  When you update the code to fix a bug, the unit test is 
> right there to modify as well.

I guess that was what I was alluding to as well. I certainly agree that 
having the tests that close is handy for users of a module. The extra 
point you make is that the unittest approach is also easier for the 
maintainer, which is fair enough.

>
> The whole point of unittests are, if they are not easy to do and 
> conveniently located, people won't do them.  You may have a really 
> good system and good coding practices that allows you to implement 
> tests the way you do.  But I typically will forget to update tests 
> when I'm updating code.  It's much simpler if I can just add a new 
> line right where I'm fixing the code.

In practice I find that unit tests are often big and complex, and they 
deserve to be separate programs in their own right. The main exception 
to this is low-level libraries (like phobos?).

>
>> What I do instead is put unit tests into separate modules, and use a 
>> custom build system that compiles, links AND executes the unit test 
>> modules (when out of date of course). The build fails if a test does 
>> not pass.
>>
>> The separation of the test from the code under test has plenty of 
>> advantages and no down-side that I can see - assuming you use a build 
>> system that understands the idea. Some of the advantages are:
>> * No code-bloat or unnecessary imports.
>
> Not a real problem with version(unittest).
>
>> * Much easier to manage inter-module dependencies.
>
> Not sure what you mean here.

I mean that the tests typically have to import way more modules than the 
code under test, and separating them is a key step in eliminating 
circular imports.

>
>> * The tests can be fairly elaborate, and can serve as well-documented 
>> examples of how to use the code under test.
>
> This is not an against for unit tests, they can be this way as well.  
> Unit testing phobos takes probably a minute on my system, including 
> building the files.  They are as complex as they need to be.

Conceded - it doesn't matter where the tests are, they can be as big as 
they need to be.

As for the time tests take, an important advantage of my approach is 
that the test programs only execute if their test-passed file is out of 
date. This means that in a typical build, very few (often 0 or 1) tests 
have to be run, and doing so usually adds way less than a second to the 
build time. After every single build (even in release mode), you know 
for sure that all the tests pass, and it doesn't cost you any time or 
effort.

>
>> * Since they only execute during the build, and even then only when 
>> out of date, they can afford to be more complete tests (ie use plenty 
>> of cpu time)
>
> IMO unit tests should not be run along with the full application.  I'd 
> suggest a simple unit test blank main function.  I think even dmd (or 
> rdmd?) will do this for you.
>
> There is no requirement to also run your application when running unit 
> tests.

That is my point exactly. I don't run tests as part of the application - 
the tests are separate utilities intended to be run automatically by the 
build tool. They can also be run manually to assist in debugging when 
something goes wrong.

>
>> * If the code builds, you know all the unit tests pass. No need for a 
>> special unittest build and manual running of assorted programs to see 
>> if the tests pass.
>
> This all stems from your assumption that you have to run unittests 
> along with your main application.
>
> When I use D unit tests, my command line is:
>
> <command to build library/app> unittests
>
> e.g.
>
> make unittests
>
> No special build situations are required.  You can put this into your 
> normal build script if you wish (i.e. build 2 targets, one unit tested 
> one and one release version).
>
> i.e.:
>
> all: unittests app
>
>> * No need for special builds with -unittest turned on.
>
> Instead you need a special build of other external files?  I don't see 
> any advantage here -- on one hand, you are building special extra 
> files, on the other hand you are building the same files you normally 
> build (which you should already have a list of) with the -unittest 
> flag.  I actually find the latter simpler.
>
> -Steve

The difference in approach is basically this:

With unittest, tests and production code are in the same files, and are 
either built together and run together (too slow); or built separately 
and run separately (optional testing).

With my approach, tests and production code are in different files, 
built at the same time and run separately. The build system also 
automatically runs them if their results-file is out of date (mandatory 
testing).


Both approaches are good in that unit testing happens, which is very 
important. What I like about my approach is that the tests get run 
automatically when needed, so regressions are discovered immediately (if 
the tests are good enough). I guess you could describe the difference as 
automatic incremental testing versus manually-initiated batch testing.


-- 
Graham St Jack