D compiler benchmarks

[Isaac Gouy]

Robert Clipsham Wrote:

> Georg Wrede wrote:
> > Suppose you have run the same program very recently before the test. 
> > Then the executable will be in memory already, any other files it may 
> > want to access are in memory too.
> > 
> > This makes execution much faster than if it were the first time ever 
> > this program is run.
> > 
> > If things were deterministic, then you wouldn't run several times and 
> > average the results, right?
> 
> Ok, I will rerun the tests later today and disregard the first test. I 
> may also take the minimum value rather than taking an average (thanks to 
> Bill Baxter for this idea).


As you're re-inventing functionality that's in the benchmarks game measurement scripts, let me suggest that there are 2 phases involved:

1) record measurements
2) analyze measurements

As long as you keep the measurements in the order they were made in and keep the measurements for each different configuration in their own file, you can decide to do different selections from those measurements at some later date.

You can throw away the first measurement or not, you can take the fastest or the median, you can ... without doing new measurements.

As you are only trying to measure a couple of language implementations, measure them across a dozen different input values rather than one or two - leaving the computer churning overnight will help keep your home warm :-)