Feature suggestion: in-place append to array

[Steven Schveighoffer]

On Wed, 31 Mar 2010 17:57:07 -0400, Mike S  
<mikes at notarealaddresslololololol.com> wrote:

> Steven Schveighoffer wrote:
>>  You are correct, setCapacity ensures that *at least* the given number  
>> of elements will be available for appending.
>>  I planned on making the function a property (but a bug would not allow  
>> that), the original intended usage was:
>>  a.capacity = 10000;
>>  Reserve doesn't work in this context.  Can you come up with a name  
>> that does?
>>  I'll bring up reserve (as a function) as an alternative on the phobos  
>> mailing list, and see what people say.  I kind of liked the  
>> setter/getter idea, but you make a good point.
>>  -Steve
>
> Sorry if resurrecting this thread is against netiquette, but it caught  
> my eye, and this is my first newsgroup post in years. ;)
>
> Anyway, is there any compelling reason why setCapacity or modifying  
> a.capacity should allocate a nondeterministic amount of storage?

What do you mean by nondeterministic?  It's very deterministic, just not  
always easy to determine ;)  However, given enough context, it's really  
easy to determine.

> Depending on the application, programmers might require strict control  
> over memory allocation patterns and strict accounting for allocated  
> memory.  Game programmers, especially console game programmers, tend to  
> strongly prefer deterministic allocation patterns, and nondeterminism is  
> one of the [several] common complaints about the C++ STL  
> (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2271.html is a  
> good resource on these kind of issues).  In the case of D (which I'm  
> considering learning), this is especially important for dynamic arrays,  
> partly because they're so useful by themselves, and partly because they  
> may form the backbone of custom containers.

The amount of memory given is determined by the GC, and ultimately by the  
OS.  The currently supported OSes allocate in Page-sized chunks, so when  
you allocate any memory from the OS, you are allocating a page (4k).  Most  
likely, you may not need a whole page for the data you are allocating, so  
the GC gives you more finely sized chunks by breaking up a page into  
smaller pieces.  This strategy works well in some cases, and can be  
wasteful in others.  The goal is to strike a balance that is "good enough"  
for everyday programming, but can be specialized when you need it.

If you want to control memory allocation yourself, you can always do that  
by allocating page-sized chunks and doing the memory management on those  
chunks yourself.  I do something very similar in dcollections to speed up  
allocation/destruction.

> Whereas it's easy to add "smart nondeterministic" behavior to a  
> deterministic setCapacity function by providing a wrapper, ordinary  
> language users can't do the opposite.  Because of this, and because  
> dynamic arrays are so central to the D language, a nondeterministic  
> setCapacity function may deter game programmers, especially console  
> programmers, from adopting D.
>
> Assuming you see this post, what are your thoughts here?

I think D has deterministic allocation, and better ability than C++ to  
make custom types that look and act like builtins.  Therefore, you can  
make an array type that suits your needs and is almost exactly the same  
syntax as a builtin array (except for some things reserved for builtins,  
like literals).  Such a thing is certainly possible, even with using the  
GC for your allocation.

BTW, I made the change to the runtime renaming the function previously  
known as setCapacity to reserve.  It won't be a property, even if that bug  
is fixed.

-Steve