Array append performance

[superdan]

Jerry Quinn Wrote:

> dsimcha Wrote:
> 
> > I just ran into this yesterday porting C++ code that used push_back() heavily to
> > D.  Ended up rewriting the inner loops to pre-allocate.  The D code is now
> > actually faster than the C++ code, but the inner loops are less readable.  For a
> > whopping overhead of 4 extra bytes per array (on 32-bit), why not include the
> > 
> > Other than the 4 bytes of extra overhead, does anyone see any downside to
> > including a capacity field?  Also, is there some situation I'm not thinking of
> > where the 4 bytes actually matter more than the speed gains in question?
> 
> For starters, you need 8 bytes on 64 bit systems.  I suspect a more important reason might be making slice operations more difficult to implement correctly.
> 
> That said, I think I still favor it, since I find I need to do iterative appends to arrays fairly often.  Otherwise, it noticably restricts the places I can use built-in arrays.

sorry to always be the one who puts the moose on the table. y'all who want to make capacity a field in the array are off base.

today each slice has length and data. why. because all a slice is *is* length and data. it's a view of a portion of memory. the slice don't care where memory came from. capacity is part of the memory that slices come from. it would be shared by several slices. and all of a sudden you realize capacity can't be a field in the slice.

can't be a field just before the actual data in the slice either. for obvious reasons.

> An alternative might be a convenient simple library wrapper that provides these kinds of dynamic operations, while still allowing easy access to the underlying array.  I'm thinking prehaps std.array might include an appender object that you create, append to, and then grab the underlying array from, once you're done using it.  It's not hard to write, but I'm sure it would be a welcome addition to the library, even if core arrays don't give us this.

that makes more sense. a sort of in-memory stream thing.