Writing a unit test for a singleton implementation

[Idan Arye]

On Thursday, 16 May 2013 at 21:10:52 UTC, Dmitry Olshansky wrote:
> 16-May-2013 02:21, Idan Arye пишет:
>> But
>> I *want* to (almost)cause a race condition here
>
> You can't "cause" a race condition - it's there by definition 
> of code.
> What you can cause is a data corruption that happened with one 
> bad interleave of reads/writes because it was possible.

That's why I added the "(almost)". A race condition happens when 
one thread reads a variable and writes it back based on the old 
value, and between that read and write another thread writes to 
that variable. By adding a `sleep()` between that read and write, 
I can conjure my own race condition. Ofcourse, the race condition 
will never happen, because the singleton's implementation uses 
synchronization to prevent it - but that's the whole point of my 
unit test, to check that the implementation is doing the 
synchronization correctly.

> I'm not seeing this test do anything useful anyway - you know 
> there is a race condition and then you try to test to see if it 
> works as just as if there is none.
>
> It's more like unscrewing a bunch of bolts and seeing if the 
> car manages it to the other town. It might or not depending on 
> how the driver rides it and on a ton of other chances - truth 
> be told it's blind luck almost. Even if you unscrew the same 
> bolts exactly the same way the 2nd car will not ride exactly as 
> long a distance as 1st one.

But if I have a system in my car that allows it to keep traveling 
even after unscrewing some bolts, and I want to test it, the way 
to do it is to unscrew those bolts and try to drive it to the 
other town. If I can't get to the other town, that means that 
system failed.

That's what I'm doing here - I have a system that prevents a race 
condition in the singleton's instantiation, and in order to test 
that system I try to force a race condition in the singleton's 
instantiation.

> And if your PC was compressing video or serving some HTTP 
> you'll have even less no matter how you try to run them I 
> guess...

OK, I just tested it by playing for video files at once, and the 
accuracy dropped from 100% to around 96%.

Still, the point of unit tests is to make sure that code changes 
do not break old code. Most of them are super trivial, because 
many of the bugs they prevent are also super trivial - the kind 
of bugs that make you want to hit yourself for being so stupid 
you didn't notice them before. If the library or system have a 
good set of unit tests, a programmer that runs them can catch 
those bugs happening in trivial examples right after they are 
introduced.

So, if a Phobos\druntime\dmd developer somehow manages to change 
something that breaks my singleton implementation, the next time 
they run the unit tests there is over 90% chance that unit test 
will catch the bug even if they put a heavy burden on their 
machine - and if they don't put such a heavy burden while 
developing, those chances climb very near to 100%.

So yea, my unit test is not 100% accurate in worst case scenarios 
- but it still does it's job.

> But if you like it I think Thread.yield will work just as well, 
> it will cause threads to do the context switch.

I tested both. `Thread.yield()` gives around 92% accuracy, while 
`Thread.sleep(dur!"msecs"(0))` gives 100% accuracy(when I don't 
play 4 video files at once). I have no idea why this happens, but 
it happens.

> Sleep doesn't guarantee it. It causes context switch though and 
> that might be what you want. Maybe creating a bunch of threads 
> as suspended and then waking them all up could get close to 
> that.

That's what I did in the second version - I suspended all the 
threads using `core.sync.barrier`, and the barrier took care to 
resume them all at once. This allowed me to use a 0ms sleep - but 
the call to `sleep()` is still required.

> Another problem is about expecting something definite out of 
> race condition. Yes, here you are getting away with single 
> atomic read/write of pointer simply because of x86 architecture.

How is it an atomic read/write if I call `sleep()` *between* the 
read and the write?

> Technically what you'll can see with a race condition is 
> undefined (not speaking of this simple example on x86 that IMO 
> is pointless anyway).
> Thus trying to catch it in more complex situation will require 
> more then just putting a sleep(xyz) before a function call.
>
> Imagine trying to test a lock-free collection ? You still need 
> to lauch many threads and somehow try to schedule them funky. 
> Even then I don't see how 1 single-shot can be reliable there.

True - my testing method is not possible with lock-free 
collections, since I can't put a `sleep()` call inside an atomic 
operation. But I *can* put a sleep call inside a `synchronized` 
block(or more precisely, inside a constructor that is being 
invoked inside a synchronized block), so my testing method does 
work for my case.