Array append performance

[bearophile]

Steven Schveighoffer:
> I see nothing wrong with having a library solution to this. [...]
> The problem is that many times I don't append to an array or slice, why 
> should I have to accept the cost of an extra 4-8 byte field for every slice 
> and array that isn't going to change size (which I would argue is used more 
> often)?

Probably a library solution makes you use even more memory (when you want to use it), and makes dynamic arrays less handy to use in D (but it may be better than nothing).

Array append is an operation rather common in higher-level programs, for example in Python it's one of the most common operations, while the up-front creation of empty arrays is quite uncommon. I don't know how much common the push_back is in C++ programs.

(In successive benchmarks I have seen that array append of integers with Psyco is about as fast as appending integers in D. A difference is that in Python integers are objects, around 20 bytes long).

My idea was to do this change in a D branch, and see how programs like your ones behave in the modified version: if your code at runtime has to use 100-200 bytes more than before, then I think this price may be accepted on modern PCs. On the other hand a large number of such size_t fields may be wasted when you manage tons of short strings, or when you manage many smallish nD arrays. That problem with nD arrays can be solved with a library that manages rectangular nD arrays, that avoids wasting of space for the lengths of each row/etc. This means that another solution is to do the opposite thing, instead of adding a library support for efficiently extensible arrays, such support may be of dynamically allocated fixed-size arrays... :-)

Bye,
bearophile