Optimization fun

[Dmitry Olshansky via Digitalmars-d]

07-Nov-2014 09:05, H. S. Teoh via Digitalmars-d пишет:
> On Thu, Nov 06, 2014 at 02:58:16PM -0800, H. S. Teoh via Digitalmars-d wrote:
> [...]
>> 1) The GC could use some serious improvement: it just so happens that
>> the solver's algorithm only ever needs to allocate memory, never
>> release it (it keeps a hash of visited states that only grows, never
>> shrinks).  The profiler indicated that one of the hotspots is the GC.
>> So I added an option to the program to call GC.disable() if a command
>> line option is given. The result is a 40%-50% reduction in running
>> time.
> [...]
>
> Yet more GC drama unfolded tonight: while testing my new GC-disabling
> option on several other puzzles, one instance ate up my PC's RAM at an
> astonishing rate, causing the system to almost freeze up from the
> ensuing thrashing. A little investigation revealed that there are
> several instances of excessive GC load caused by unnecessary
> (re)allocations.


That's the problem with profilers:
	they say what takes time but not why :)

Often I find myself looking at memcpy at the top of the list, so obvious 
the "textbook" answer is to optimize memcpy ;) In contrast it should be 
read as "you seem to do excessive copying of data".

> The moral of the story, is that code like the following should raise big
> red warning flags:
>
> 	struct MyData(T) {
> 		T[] data;
> 		T pop() {
> 			...
> 			data.length--; // <--- yikes
> 			...
> 		}
> 		void push(T t) {
> 			...
> 			data ~= t; // <--- yikes
> 			...
> 		}
> 	}

Well known in the inner circles performance bug. Sadly most of D 
programmers are bound to hit it once in the while.

Memory safety "by default" has its costs.
Though I'd just call mmap, far more predictable then these pesky memory 
allocators + no need to reallocate if reserved range is large enough.


-- 
Dmitry Olshansky