[Issue 5813] [patch] std.array.Appender has severe performance and memory leak problems.

[d-bugmail at puremagic.com]

http://d.puremagic.com/issues/show_bug.cgi?id=5813



--- Comment #10 from Rob Jacques <sandford at jhu.edu> 2011-12-02 12:35:48 PST ---
(In reply to comment #9)
> I did some benchmarks with this and some other variants.
> 
> At least for small strings, this code performs worse than the current appender.
> Judging by the description it's optimized for large blocks of data, but in the
> case of many small strings it can perform as bad as 50% worse than the current
> appender for typical cases (e.g. building a few KB of HTML).
>
> Here is my test code:
> 
> http://dump.thecybershadow.net/a05a2c4dc7cd2a8e21b3a447c7eff150/test2.d
> http://dump.thecybershadow.net/eff5c7ef81e18bf75d8462ffe16a52e4/appender.d
> 
> I was able to get 25% more performance for my test case from the standard
> appender by adding a method that accepts multiple arguments (which preallocates
> only once). My code is a hack, but it would be nice to see something like that
> in the official appender.

Thank you for the additional benchmarks. The primary purpose of my code was to
optimize for memory usage, which in turn optimizes for computation/performance
(i.e. number of main memory copies and memory allocations). And, I also
optimized appender for use as a dynamic buffer. But this is all big-O stuff.

I did include a short string test in my benchmark set while I was optimizing
the code; my first implementations (never posted) did have some major slow
downs, which I fixed by re-working the put routine. So there still might still
be some little-o issues there. And I will definitely add a void put(U...)(U
items) overload.

I don't think it would affect this benchmark in any ways, but my appender is
somewhat dependent on my patch to put (Issue 5233).

I've run your benchmarks and played with them a little bit. Running the
benchmarks sequentially is definitely generating GC warm-up issues. I've
recommend running them cold in the future, though the qualitative results are
the same and cold runs are the ideal situation for the old appender.

I want to rewrite my __ctfe version appender, since Don added support for
structs and new, so I'll also look at the short string performance of appender
while I'm at it I'll do some profiling and look into this some more. I'd really
like for appender to be the right solution for anything bigger than `hello` ~
`world`.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------