D compiler benchmarks

[Georg Wrede]

Robert Clipsham wrote:
> Georg Wrede wrote:
>> Robert Clipsham wrote:
>>> Hi all,
>>>
>>> I have set up some benchmarks for dmd, ldc and gdc at 
>>> http://dbench.octarineparrot.com/.
>>>
>>> There are currently only 6 tests all from 
>>> http://shootout.alioth.debian.org/gp4/d.php. My knowledge of phobos 
>>> is not great enough to port the others to tango (I've chosen tango as 
>>> ldc does not support phobos currently, so it make sense to choose 
>>> tango as all compilers support it). If you would like to contribute 
>>> new tests or improve on the current ones let me know and I'll include 
>>> them next time I run them.
>>>
>>> All source code can be found at 
>>> http://hg.octarineparrot.com/dbench/file/tip.
>>>
>>> Let me know if you have any ideas for how I can improve the 
>>> benchmarks, I currently plan to add compile times, size of the final 
>>> executable and memory usage (if anyone knows an easy way to get the 
>>> memory usage of a process in D, let me know :D).
>>
>> The first run should not be included in the average.
> 
> Could you explain your reasoning for this? I can't see why it shouldn't 
> be included personally.

Suppose you have run the same program very recently before the test. 
Then the executable will be in memory already, any other files it may 
want to access are in memory too.

This makes execution much faster than if it were the first time ever 
this program is run.

If things were deterministic, then you wouldn't run several times and 
average the results, right?