D is 71% faster than D - or why benchmarking often returns odd results (Re: Coolest D features)

[Georg Wrede]

Thomas Kuehne wrote:
> Waldemar schrieb am 2006-12-28:
>> That may be but this specific C compiler is most likely gcc on
>> Linux  or VS C++ on Windows. The D compiler is probably dmd.

Yes, most likely. This or that.

>> It's a bit shocking to see a 50% difference. Is there information
>> which compilers were used?

I suspect no foul play, but the Western Science Dogma of Repeatability 
simply demands that we be provided with enough information to "try this 
at home".

>> And is  there any reason to believe the specifics of the benchmark
>> could produce such a  wide difference?
> 
> Comparing SciMark2 (svn://dstress.kuehne.cn/benchmark/scimark) compiled
> with GDC-0.20 and DMD-0.178 suggests that D is 71% faster than D <g>

While this quip is most entertaining, many in the audience may miss the 
point. Which, unfortunately, shrouds the issue altogether.

> GDC-0.20:
> DMD-0.178:
> 	Using       2.00 seconds min time per kernel.
> 	Using       2.00 seconds min time per kernel.
> 	Composite Score:          366.95
> 	Composite Score:          213.89
> 	FFT             Mflops:   552.68    (N=1024)
> 	FFT             Mflops:   240.84    (N=1024)
> 	SOR             Mflops:   338.06    (100 x 100)
> 	SOR             Mflops:   347.83    (100 x 100)
> 	MonteCarlo      Mflops:    63.91
> 	MonteCarlo      Mflops:    48.63
> 	Sparse matmult  Mflops:   346.75    (N=1000, nz=5000)
> 	Sparse matmult  Mflops:   107.08    (N=1000, nz=5000)
> 	LU              Mflops:   533.33    (M=100, N=100)
> 	LU              Mflops:   325.08    (M=100, N=100)
> 
> 
> The most important question of benchmarking:
 >
> What do you compare, and why do you compare it?

An ancient folk wisdom from Sipoo, Finland: "A man with clean flour, 
never has to hide his palms."