Andrei Alexandrescu needs to read this

[Jonathan Marler]

On Wednesday, 23 October 2019 at 21:37:26 UTC, welkam wrote:
> I watched many of his talks and he frequently talks about 
> optimization that produce single digits % of speed up in 
> frequently used algorithms but doesnt provide adequate prove 
> that his change in algorithm was the reason why we see 
> performance differences. Modern CPUs are sensitive to many 
> things and one of them is code layout in memory. Hot loops are 
> the most susceptible to this to the point where changing user 
> name under which executable is run changes performance. A paper 
> below goes deeper into this.
>
> Producing Wrong Data Without Doing Anything Obviously Wrong!
> https://users.cs.northwestern.edu/~robby/courses/322-2013-spring/mytkowicz-wrong-data.pdf

That's why Andrei says "always measure". He understands how 
complex modern CPUs are and that it's basically pointless to try 
to predict performance.  He has some good talks where he shows 
that one of the biggest causes of performance problems is not 
understanding how the processor cache works, but his point was 
that you'll never be able to theoretically predict the 
performance of hardware in today's world.  Always measure.

What I find funny is that there are alot of clever tricks you can 
do to make your code execute less operations, but with modern 
CPUs it's more about making your code more predictable so that 
the cache can predict what to load next and which branches you're 
more likely to take.  So in a way, as CPUs get smarter, you want 
to make your code "dumber" (i.e . more predictable) in order to 
get the best performance.  When hardware was "dumber", it was 
better to make your code smarter.  An odd switch in paradigms.