if(false) vs if(isNaN) benchmark

Joseph Rushton Wakeling joseph.wakeling at webdrake.net
Thu Jul 4 08:29:52 PDT 2013


On 07/04/2013 04:05 PM, monarch_dodra wrote:
> Holly crap, I didn't see that. Don't do that. The problem if you do that is that
> on a lot of machines, the time it takes to do 1 iteration is shorter than the
> granularity of the clock. For example, on windows, it is typically 15 ms, which
> is HUGE! I think *nix is more precise, but not *that* precise either. If you are
> unlucky, the system clock may increment between iterations, meaning you'll miss
> out on that time.
> 
> In particular, if your tests last 30ms, it can make a big difference. I usually
> try to make sure my tests last at least a whole second. I also try to add an
> entire second of warm up, the time it takes for the OS to get its caching done.
> 
> I also try to run the tests in such a way that each case is tested several times
> alternatively. Sometimes, you'd be surprised with the result.

You're absolutely right.  With the watch stop/start placed outside the foreach
loop, the times measured are cut dramatically.  The difference is still in the
range of about 2-5 ms.

To avoid the problem of the stopwatch increments being too small relative to
test time, I've put the watch stop/start outside the foreach loops, and made the
foreach loops iterate over a cycle of 100 times the underlying array.

Typical results with dmd -O -release -inline -noboundscheck:

    Time for 100000000 if(boolean)'s with 90000000 counts of true:
        1.3926e+06 microseconds [0.013926 per test].
    Time for 100000000 if(isNaN)'s with 90000000 counts of true:
        1.53494e+06 microseconds [0.0153494 per test].
    Total difference: -142345 [-0.00142345 per test].

Typical results with gdmd -O -release -inline -noboundscheck:

    Time for 100000000 if(boolean)'s with 90000000 counts of true:
        899075 microseconds [0.00899075 per test].
    Time for 100000000 if(isNaN)'s with 90000000 counts of true:
        1.33906e+06 microseconds [0.0133906 per test].

    Total difference: -439986 [-0.00439986 per test].

Typical results with ldmd2 -O -release -inline -noboundscheck:

    Time for 100000000 if(boolean)'s with 90000000 counts of true:
        912115 microseconds [0.00912115 per test].
    Time for 100000000 if(isNaN)'s with 90000000 counts of true:
        1.27287e+06 microseconds [0.0127287 per test].

    Total difference: -360753 [-0.00360753 per test].

Now if we miss off -inline:

dmd -O -release -noboundscheck:
    Time for 100000000 if(boolean)'s with 90000000 counts of true:
        1.95245e+06 microseconds [0.0195245 per test].
    Time for 100000000 if(isNaN)'s with 90000000 counts of true:
        2.31204e+06 microseconds [0.0231204 per test].

    Total difference: -359596 [-0.00359596 per test].

gdmd -O -release -noboundscheck:
    Time for 100000000 if(boolean)'s with 90000000 counts of true:
        905912 microseconds [0.00905912 per test].
    Time for 100000000 if(isNaN)'s with 90000000 counts of true:
        1.3392e+06 microseconds [0.013392 per test].

    Total difference: -433283 [-0.00433283 per test].

ldmd2 -O -release -noboundscheck:
    Time for 100000000 if(boolean)'s with 90000000 counts of true:
        900240 microseconds [0.0090024 per test].
    Time for 100000000 if(isNaN)'s with 90000000 counts of true:
        1.2728e+06 microseconds [0.012728 per test].

    Total difference: -372557 [-0.00372557 per test].

Missing off -noboundscheck seems to make no difference with gdmd or ldmd2 but
creates a bit of slowdown for dmd:

dmd -O -release:
    Time for 100000000 if(boolean)'s with 90000000 counts of true:
        2.68152e+06 microseconds [0.0268152 per test].
    Time for 100000000 if(isNaN)'s with 90000000 counts of true:
        3.00835e+06 microseconds [0.0300835 per test].

    Total difference: -326823 [-0.00326823 per test].

Lastly, I tried replacing the non-NaN double value with double.infinity instead
of 1.0.  It makes no appreciable difference to the above results, at least with
all the optimization flags enabled (I didn't test it without).

Code attached again. :-)

Anyway, in summary, it looks like the extra time cost of using an if(isNaN) as
opposed to an if(boolean) is at most about 5e-9 seconds, and this is consistent
pretty much regardless of the implementation flags used (even rdmd with no
optimization flags reports a difference per test of about 0.003 microseconds).
-------------- next part --------------
A non-text attachment was scrubbed...
Name: isnan.d
Type: text/x-dsrc
Size: 1331 bytes
Desc: not available
URL: <http://lists.puremagic.com/pipermail/digitalmars-d-learn/attachments/20130704/24e68460/attachment-0001.d>


More information about the Digitalmars-d-learn mailing list