Beginner's Comparison Benchmark

[H. S. Teoh]

On Wed, May 06, 2020 at 09:59:48AM +0000, welkam via Digitalmars-d-learn wrote:
> On Tuesday, 5 May 2020 at 20:29:13 UTC, Steven Schveighoffer wrote:
> > the optimizer recognizes what you are doing and changes your code
> > to:
> > 
> > writeln(1_000_000_001);
> > 
> Oh yes a classic constant folding. The other thing to worry about is
> dead code elimination. Walter has a nice story where he sent his
> compiler for benchmarking and the compiler figured out that the the
> result of the calculation in benchmark is not used so it deleted the
> whole benchmark.

I remember one time I was doing some benchmarks between different
compilers, and LDC consistently beat them all -- which is not
surprising, but what was surprising was that running times were
suspiciously short.  Curious to learn what magic code transformation LDC
applied to make it run so incredibly fast, I took a look at the
generated assembly.

Turns out, because I was calling the function being benchmarked with
constant arguments, LDC decided to execute the entire danged thing at
compile-time and substitute the entire function call with a single
instruction that loaded its return value(!).

Another classic guffaw was when the function return value was simply
discarded: LDC figured out that the function had no side-effects and its
return value was not being used, so it deleted the function call,
leaving the benchmark with the equivalent of:

	void main() {}

which, needless to say, beat all other benchmarks hands down. :-D

Lessons learned:

(1) Always use external input to your benchmark (e.g., load from a file,
so that an overly aggressive optimizer won't decide to execute the
entire program at compile-time); 

(2) Always make use of the return value somehow, even if it's just to
print 0 to stdout, or pipe the whole thing to /dev/null, so that the
overly aggressive optimizer won't decide that since your program has no
effect on the outside world, it should just consist of a single ret
instruction. :-D


T

-- 
This is not a sentence.