if(false) vs if(isNaN) benchmark

[monarch_dodra]

On Thursday, 4 July 2013 at 13:43:13 UTC, Joseph Rushton Wakeling 
wrote:
> On 07/04/2013 03:05 PM, monarch_dodra wrote:
>> Me thinks the test is biased with answering true to isNan. You 
>> should first
>> initialize your arrays with random [true/false] | [nan/nonan] 
>> value. In
>> particular, it is more important for isNan to answer "No" as 
>> fast as possible,
>> rather than actually find nan's. (common, use case is checking 
>> that a number is
>> *not*) nan. I'd say a mix of 90% rands + 10% Nan's would be 
>> representative?
>
> Quite right -- the case of isNaN being false is the case that 
> needs to be really
> quick.
>
>> Also, there might be optimizations in bool iteration over 
>> double iteration given
>> their size, as well as the fact that the "first" test is 
>> typically faster.
>
> I'd assumed that putting the start/stop of the stopwatch inside 
> the loop would
> address that, but apparently not ...

Holly crap, I didn't see that. Don't do that. The problem if you 
do that is that on a lot of machines, the time it takes to do 1 
iteration is shorter than the granularity of the clock. For 
example, on windows, it is typically 15 ms, which is HUGE! I 
think *nix is more precise, but not *that* precise either. If you 
are unlucky, the system clock may increment between iterations, 
meaning you'll miss out on that time.

In particular, if your tests last 30ms, it can make a big 
difference. I usually try to make sure my tests last at least a 
whole second. I also try to add an entire second of warm up, the 
time it takes for the OS to get its caching done.

I also try to run the tests in such a way that each case is 
tested several times alternatively. Sometimes, you'd be surprised 
with the result.

>> I think you should create an array of S{bool; double;} So that 
>> the iteration is
>> not biased either.
>> 
>> I'd add a dummy "warmup" loop of 10% of the iterations too, 
>> just 'cause.
>
> See attached.  I haven't put in the dummy warmup loop, but as 
> is, there's
> consistently less than 2ms difference between the times 
> reported for all
> compilers.  If I use rdmd without optimizations, the difference 
> is slightly
> larger at 3-5ms.

Cool. Even gdmd? Your first trial showed 150 vs 30 ms.

> So, on the basis of that I think I feel happy about using isNaN 
> as a test,
> especially as it lets me remove an otherwise superfluous 
> variable from
> RandomSample.  Thanks for the advice! :-)

No problem. Seeing people make tests is always interesting. You 
could also throw in a few "inf" in there, as inf is *really* 
close binary wise to NaN: It will pass the first "test", and not 
the second. But I think inf's are even rarer then NaN's, so...

Also, those tests are with inline? Could you report the times 
without please (for my own curiosity)?