Profiling graph library

[Joseph Rushton Wakeling]

On 07/31/2013 02:31 PM, bearophile wrote:
> Then I suggest to try ldc2 too.

I haven't profiled with ldc2 yet (I will) but the broad speed tendencies are
similar.  It's faster but not so much faster that I can assume the speed hit
isn't still there.

> My suggestion is to copy the implementation of those ranges in a test module,
> and try to understand what are the causes of the GC activity you see.

I'll look into that.  Thanks for the suggestion. :-)

> If this resulting array is used only for a short period of time:
> 
> return sequence.map!(x => x.func).array;
> 
> 
> then perhaps you can perform an array allocation in a constructor, static
> constructor, or you could allocate a fixed sized array inside the body of the
> graph, etc. Let's say this array is named "buf".
> 
> Then with code like:
> 
> auto left = sequence.map!(x => x.func).copy(buf[]);
> return buf[...];
> 
> left tells you how much is left, so you can slice the buf and return it.
> Somewhere you also have to safe the length of the used buf.
> If the resulting array is used only for a moment, this will not cause troubles
> in a single thread program. The point is to remove the memory allocation caused
> by array().
> 
> An alternative design comes from Tango, instead of allocating the buffer
> somewhere, your function takes an optional buffer, and uses it unless it's too
> much small for the purpose, otherwise it allocates with array() or something else.

Oh, I see.  Yes, that makes sense and could be useful.

Actually, something like this (but not done as elegantly as your example) was
the reason behind the cached version of the graph.  I did think of just
returning a small buffer with the neighbours in, but I was concerned that if
someone wanted to keep using the resulting array over a longer period of time,
errors could creep in.

So, I came up with the full cache that is memory-heavy but safer as any copies
you take will remain valid so long as the graph itself is not altered.

But most likely the best compromise solution is as you suggest a buffer, with
the instruction ".dup the result if you want it to be persistent".