Strange measurements when reproducing issue 5650

[Steven Schveighoffer]

On Fri, 27 Apr 2012 10:09:18 -0400, SomeDude <lovelydear at mailmetrash.com>  
wrote:

> On Wednesday, 25 April 2012 at 17:37:33 UTC, H. S. Teoh wrote:
>>
>> First of all, differences as small as 20ms really should be considered
>> as background noise. The exact measurements depend on a lot of
>> system-specific and environment-specific factors, such as OS memory
>> usage, CPU cache behaviour, disk activity & speed, the exact points of
>> context switches, etc.. If you really want to check for substantial
>> performance differences, you need to magnify your test case so that
>> differences are measured >5 seconds.
>>
>> Second, on my AMD hexacore 64-bit Linux system, the running time
>> consistently measures between 0.57 or 0.58 seconds for both cases. The
>> exact figure changes between runs, and as far as I can tell, there's no
>> discernible difference between the two.
>>
>> Third, to make any timing differences stand out from the background
>> noise, I increased n to 20_000_000, and both versions of the program
>> consistently runs in about 11 seconds each time. There's no discernible
>> difference between the two.
>>
>> What all this means is that a single call to writeln does not make
>> enough difference to be measurable compared to the rest of the program.
>> It doesn't mean that the version with writeln is "faster", just that the
>> difference is too small and you're probably just seeing background
>> noise. If you put the writeln inside a loop, on the other hand, you'll
>> see a big difference, because now its cost is magnified by the number of
>> times the loop runs. (Say if you put it inside a foreach(i;0..1000) at
>> the end of the program, you'll see the difference when you comment it
>> out.)
>>
>> So I'd chalk it up to inherent measurement inaccuracies.
>>
>>
>> T
>
> Thanks, indeed, for n = 5_000_000, I observe the expected result, i.e  
> the writeln version being almost 1 second slower than without.
> Below 2_000_000, though, I consistently see the opposite on my machine,  
> i.e the version with writeln being slightly faster. If it was background  
> noise, it would be about equal, no ?

writeln should not make a difference.  Even walkLength should not,  
Especially over 100 nodes.

What you are likely seeing is some weird issue with GC cleanup.

Have you tried measuring the code timings just inside main instead of the  
full execution of the program including runtime startup and shutdown?

-Steve