Review of Andrei's std.benchmark

[Manu]

On 19 September 2012 01:02, Andrei Alexandrescu <
SeeWebsiteForEmail at erdani.org> wrote:

> On 9/18/12 5:07 PM, "Øivind" wrote:
>
>> * For all tests, the best run is selected, but would it not be
>
> reasonable in some cases to get the average value? Maybe excluding the
>> runs that are more than a couple std. deviations away from the mean
>> value..
>>
>
> After extensive tests with a variety of aggregate functions, I can say
> firmly that taking the minimum time is by far the best when it comes to
> assessing the speed of a function.


The fastest execution time is rarely useful to me, I'm almost always much
more interested in the slowest execution time.
In realtime software, the slowest time is often the only important factor,
everything must be designed to tolerate this possibility.
I can also imagine other situations where multiple workloads are competing
for time, the average time may be more useful in that case.

Side question:
Running a test over and over pre-populates the cache with all associated
data after the first cycle... The cache needs to be randomised between each
cycle to get realistic results.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20120919/0c26b348/attachment-0001.html>