Unit tests called multiple times?

[Jesper]

On Friday, 12 April 2013 at 23:18:00 UTC, H. S. Teoh wrote:
> On Sat, Apr 13, 2013 at 01:08:38AM +0200, William wrote:
>> On Friday, 12 April 2013 at 23:01:55 UTC, Steven Schveighoffer
>> wrote:
>> >On Fri, 12 Apr 2013 18:52:50 -0400, William
>> ><squidkidsignup at gmail.com> wrote:
>> >
>> >>I'm compiling a medium-sized project written in D with the
>> >>-unittests flag.  I have some writeln statements in my unit
>> >>tests, for debug purposes, and I've noticed that certain unit
>> >>tests are being run more than once.  Sample output is along
>> >>these lines:
>> >>
>> >>DEBUG: unit test started...
>> >>DEBUG: loop found whatever
>> >>DEBUG: iteration successful
>> >>DEBUG: bar is equal to bar
>> >>DEBUG: unit test passed...
>> >>DEBUG: unit test started...
>> >>DEBUG: loop found whatever
>> >>DEBUG: iteration successful
>> >>DEBUG: bar is equal to bar
>> >>DEBUG: unit test passed...
>> >>DEBUG: unit test started...
>> >>DEBUG: loop found whatever
>> >>DEBUG: iteration successful
>> >>DEBUG: bar is equal to bar
>> >>DEBUG: unit test passed...
>> >>
>> >>is this standard behavior for DMD?  Is it documented 
>> >>anywhere?
>> >> It's not much of a problem, but it strikes me as rather odd,
>> >>and makes the debug output difficult to read.
>> >
>> >Are your unit tests inside templates?  If so, they are
>> >instantiated once per template.  To fix, move outside the
>> >template.
>> >
>> >This actually can be a nice feature, and at the same time
>> >horrible.  It's nice because you can parameterize your unit 
>> >tests!
>> >It's horrible because you have to be careful your unit tests 
>> >are
>> >valid for all possible instantiations!
>> >
>> >-Steve
>> 
>> Thank you!  That was exactly it.  I tend to put my unittests 
>> directly
>> after the functions/classes they test, so my opApply for an
>> implementation of n-dimensional associative arrays, (with
>> parameterized keys/values) was having its associated unittest 
>> called
>> every time the template was instantiated.  Moving it to the 
>> module
>> level fixed the problem, although now it reads as though the 
>> test is
>> intended to test the whole class, when in fact it only 
>> verifies the
>> correctness of opApply.  Still, feels good to have weird 
>> behavior
>> demystified.
>
> You could use static if to make the unittest only instantiate 
> for a
> particular template instance, e.g.:
>
> 	struct S(T) {
> 		...
> 		void method(T t) { ... }
>
> 		static if (T==int)
> 		unittest {
> 			auto s = S!int(123);
> 			s.method(123);
> 			assert( ... );
> 		}
> 	}
>
> This way, you can still keep unittests together with the 
> methods they
> test. Beware, though, that you actually instantiate S!int 
> somewhere
> outside, otherwise the unittest may never actually be 
> instantiated and
> run. :)
>
> (This does give rise to an interesting idiom of unittests 
> catered for
> specific types that only run when your code actually uses those
> particular instantiations. Hmm...)
>
>
> T

I have also noticed this, that unless you instantiate a class 
from a path from your main-function the unittest will not be run 
at all. Actually come to think of it I believe this also is valid 
for the very compilation of the code..no compilation of code if 
it's not instantiated, but does this only happen for 
template-classes?
I also have another problem with unittests, because my unittests 
would fail because exceptions was thrown and I would just get a 
completely useless message like:
rdmd -unittest c:\kode\datastructures\datastructures.d
DataStructures.ExceptionIndexOutOfBounds at c:\kode\datastructures\datastructures.d
(68): Index was out of bounds: 1 [2-3]
----------------
0x004155A8
0x00415433
0x0040294C
0x00408BA6
0x00411A48
0x0040BBE9
0x00408C48
0x765D336A in BaseThreadInitThunk
0x77119F72 in RtlInitializeExceptionChain
0x77119F45 in RtlInitializeExceptionChain

where 'ExceptionIndexOutOfBounds' is an exception I made myself 
and it is thrown when the unittests are run. The line-number 
given as a result points to the constructor of the 
'ExceptionIndexOutOfBounds'-exception and not to the unittest 
that failed.
The mem-addresses below I figure are supposed to be a stacktrace, 
but as you can see it lacks some basic information to make it 
useful.
Has anyone seen this before and what am I doing wrong? (or is 
this the designed behaviour?..i hope not).