Profiling graph library

[bearophile]

Joseph Rushton Wakeling:

> No, gdmd -O -release -inline -g -profile, and then gprof to 
> generate the profile.

Then I suggest to try ldc2 too.


> That's what I'd assumed, which was why I was so disappointed 
> that in this case
> performance was _very_ bad.  I particularly don't understand 
> the major GC hit.
>
> Or rather, I should say, I don't see why that GC hit should be 
> there, unless
> it's that there is a fault in the implementation of iota, map 
> and/or chain.

My suggestion is to copy the implementation of those ranges in a 
test module, and try to understand what are the causes of the GC 
activity you see.


> Could you expand a little on that point?
>
> I don't mean to ask you to write my code for me, it's just that 
> (like most
> non-computer-science researchers) my knowledge of programming 
> is somewhat
> specialized, and there are blank spots on the map.  So when in 
> your reply to
> John you say, "Allocate a buffer somewhere, on the stack or 
> heap", I'm still not
> sure exactly what you mean or how you'd use it.  It'd really 
> help to see a
> concrete example (it doesn't need to be tailored to the 
> use-case here).

If this resulting array is used only for a short period of time:

return sequence.map!(x => x.func).array;


then perhaps you can perform an array allocation in a 
constructor, static constructor, or you could allocate a fixed 
sized array inside the body of the graph, etc. Let's say this 
array is named "buf".

Then with code like:

auto left = sequence.map!(x => x.func).copy(buf[]);
return buf[...];

left tells you how much is left, so you can slice the buf and 
return it. Somewhere you also have to safe the length of the used 
buf.
If the resulting array is used only for a moment, this will not 
cause troubles in a single thread program. The point is to remove 
the memory allocation caused by array().

An alternative design comes from Tango, instead of allocating the 
buffer somewhere, your function takes an optional buffer, and 
uses it unless it's too much small for the purpose, otherwise it 
allocates with array() or something else.

Bye,
bearophile