Review of Andrei's std.benchmark

[Manu]

On 21 September 2012 07:45, Andrei Alexandrescu <
SeeWebsiteForEmail at erdani.org> wrote:

> As such, you're going to need a far more
>> convincing argument than "It worked well for me."
>>
>
> Sure. I have just detailed the choices made by std.benchmark in a couple
> of posts.
>
> At Facebook we measure using the minimum, and it's working for us.


Facebook isn't exactly 'realtime' software. Obviously, faster is always
better, but it's not in a situation where if you slip a sync point by 1ms
in an off case, it's all over. You can lose 1ms here, and make it up at a
later time, and the result is the same. But again, this feeds back to your
distinction between benchmarking and profiling.

 Otherwise, I think we'll need richer results. At the very least there
>> should be an easy way to get at the raw results programmatically
>> so we can run whatever stats/plots/visualizations/**output-formats we
>> want. I didn't see anything like that browsing through the docs, but
>> it's possible I may have missed it.
>>
>
> Currently std.benchmark does not expose raw results for the sake of
> simplicity. It's easy to expose such, but I'd need a bit more convincing
> about their utility.


Custom visualisation, realtime charting/plotting, user supplied reduce
function?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20120921/3bb43a2a/attachment.html>