Performance

[Russel Winder via Digitalmars-d]

On Sat, 2014-05-31 at 07:02 -0700, Andrei Alexandrescu via Digitalmars-d
wrote:
> On 5/30/14, 11:36 PM, Russel Winder via Digitalmars-d wrote:
> > As well as the average (mean), you must provide standard deviation and
> > degrees of freedom so that a proper error analysis and t-tests are
> > feasible. Or put it another way: even if you quote a mean with knowing
> > how many in the sample and what the spread is you cannot judge the error
> > and so cannot make deductions or inferences.
> 
> No. Elapsed time in a benchmark does not follow a Student or Gaussian 
> distribution. Use the mode or (better) the minimum. -- Andrei

We almost certainly need to unpack that more. I agree that behind my
comment was an implicit assumption of a normal distribution of results.
This is an easy assumption to make even if it is wrong. So is it
provably wrong? What is the distribution? If we know that then there is
knowledge of the parameters which then allow for statistical inference
and deduction.

-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder at ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel at winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder