std.benchmark is in reviewable state

[Robert Jacques]

On Mon, 26 Sep 2011 17:56:26 -0400, Andrei Alexandrescu <SeeWebsiteForEmail at erdani.org> wrote:

> On 9/26/11 8:43 AM, Robert Jacques wrote:
>> Great, but MS still recommends benchmarking be done on a single core.
>> And if MS thinks that is how benchmarking should be done, I think that's
>> how we should do it.
>>
>>  From MSDN documentation on QueryPerformanceCounter:
>> "On a multiprocessor computer, it should not matter which processor is
>> called. However, you can get different results on different processors
>> due to bugs in the basic input/output system (BIOS) or the hardware
>> abstraction layer (HAL). To specify processor affinity for a thread, use
>> the SetThreadAffinityMask function."
>
> I guess this may have relevance only on multithreaded apps, but it's
> best to do things by the book. I'll look into it.

Well, this applies to any multi-core PC; whether the app is single-threaded or multi-threaded makes no difference.

>> *sigh* By multiple runs, I mean multiple runs of the benchmark function,
>> _including_ the loops. From my post last Tuesday:
>
> I now got it, sorry for being slow. How about taking the minimum of 10
> consecutive bigloop runs while committing to the same overall timing budget?

No problem. 10 is a good default and is generally what I use.

Although, there should be at least one function which allows manual control out the number of bigloops. i.e. for the ptr+length discussion I ultimately used a lot more than 10 iterations, as this was a very precise comparison. And people wanting to test network or cold-start disk performance may only want to use a single bigloop.