std.benchmark ready for review. Manager sought after

[Jens Mueller]

Andrei Alexandrescu wrote:
> Hello,
> 
> 
> I finally found the time to complete std.benchmark. I got to a very
> simple API design, starting where I like it: one line of code.
> 
> Code is in the form of a pull request at
> https://github.com/D-Programming-Language/phobos/pull/529. (There's
> some noise in there caused by my git n00biness). Documentation is at
> http://erdani.com/d/web/phobos-prerelease/std_benchmark.html.
> 
> If reasonable and at all possible, I'd like to bump the priority of
> this proposal. Clearly D's user base is highly interested in
> efficiency, and many of the upcoming libraries have efficiency a
> virtual part of their design. So we should get std.benchmark in soon
> and require that new addition come with benchmarks for their
> essential functionality.
> 
> My vision is that in the future Phobos will have a significant
> benchmarks battery, which will help improving Phobos and porting to
> new platforms.

I come to think the same.

How come that the times based relative report and the percentage based
relative report are mixed in one result? And how do I choose which one
I'd like to see in the output.

When benchmarking you can measure different things at the same time. In
this regard the current proposal is limited. It just measures wall clock
time. I believe extending the StopWatch to measure e.g. user CPU time is
a useful addition.
In general, allowing user defined measurements would be great.
E.g. to measure the time spend in user mode.
() => {
         tms t;
         times(&t);
         return t.tms_utime;
      }

Note, that this code does not need to be portable. You can also use
version() else static assert.

Things that come to mind that I'd like to measure.
Time measurements:
  * User CPU time
  * System CPU time
  * Time spent in memory allocations
Count measurements:
  * Memory usage
  * L1/L2/L3 cache misses
  * Number of executed instructions
  * Number of memory allocations

Of course wall clock time is the ultimate measure when benchmarking.
But often you need to investigate further (doing more measurements).

Do you think adding this is worthwhile?

All in all the user interface has been greatly simplified.

Jens