The Computer Languages Shootout Game

[Isaac Gouy]

--- On Mon, 11/1/10, Walter Bright <newshound2 at digitalmars.com> wrote:

> From: Walter Bright <newshound2 at digitalmars.com>
> Subject: Re: The Computer Languages Shootout Game
> To: digitalmars-d at puremagic.com
> Date: Monday, November 1, 2010, 1:39 PM
> Isaac Gouy wrote:
> >> Nobody would believe benchmarks on the D web site.
> Heck, I don't believe
> >> any benchmarks published by the developers of any
> language.
> > When you publish the source code of the programs, and
> the compile and build
> > logs, and the compiler and linker versions, and the OS
> the measurements were
> > made on, ... others don't have to just "believe"
> because they can try to
> > confirm the measurements for themselves.
> 
> 
> It's true, they can, but they don't. I have nearly 30 years
> of experience with this.

It doesn't matter whether they try to confirm the measurements for themselves or not - what matters is that they are provided with the all the information required to do so.


I only have 5 years experience publishing the measurements for the benchmarks game - and I've come across a handful of people who did try to confirm the measurements for themselves.

(The most interesting example compared a couple of language implementations on one particular task but measured at 2 dozen different input values. That nicely demonstrated that the same language implementation wasn't always faster across all the input values. The 3 different input values shown on the benchmarks game isn't usually enough to demonstrate that kind of thing.)