Feature suggestion: in-place append to array

[Mike S]

Steven Schveighoffer wrote:
  > What do you mean by nondeterministic?  It's very deterministic, just 
not
> always easy to determine ;)  However, given enough context, it's really 
> easy to determine.

When I say deterministic, I'm referring to determinism from the user's 
point of view, where the allocation behavior is affected solely by the 
parameter (the size request, e.g. 10000 objects) and not by some kind of 
  internal state, hidden context, or arcane black magic. :p

  > The amount of memory given is determined by the GC, and ultimately by
> the OS.  The currently supported OSes allocate in Page-sized chunks, so 
> when you allocate any memory from the OS, you are allocating a page 
> (4k).  Most likely, you may not need a whole page for the data you are 
> allocating, so the GC gives you more finely sized chunks by breaking up 
> a page into smaller pieces.  This strategy works well in some cases, and 
> can be wasteful in others.  The goal is to strike a balance that is 
> "good enough" for everyday programming, but can be specialized when you 
> need it.

That's understandable, and it makes sense that the actual memory being 
allocated would correspond to some chunk size.  It's really just opaque 
black box behavior that poses a problem; if users are given well-defined 
guidelines and chunk sizes, that would work just fine.  For instance, a 
spec like, "reserve a multiple of 512 bytes and that's exactly what you 
will be given," would allow users to minimize wastefulness and know 
precisely how much memory they're allocating.

> 
> If you want to control memory allocation yourself, you can always do 
> that by allocating page-sized chunks and doing the memory management on 
> those chunks yourself.  I do something very similar in dcollections to 
> speed up allocation/destruction.
> 
> <snip>
> 
> I think D has deterministic allocation, and better ability than C++ to 
> make custom types that look and act like builtins.  Therefore, you can 
> make an array type that suits your needs and is almost exactly the same 
> syntax as a builtin array (except for some things reserved for builtins, 
> like literals).  Such a thing is certainly possible, even with using the 
> GC for your allocation.

That parallels what game devs do in C++:  They tend to use custom 
allocators a lot, and they're likely to follow the same basic strategy 
in D too, if/when it becomes a suitable replacement.  I'm still just 
browsing though, and I'm not all that familiar with D.  If you can't 
actually use the built-in dynamic arrays for this purpose, how difficult 
would it be to reimplement a contiguously stored dynamic container using 
custom allocation?  I suppose you'd have to build it from the ground up 
using a void pointer to a custom allocated block of memory, right?  Do 
user-defined types in D have any/many performance disadvantages compared 
to built-ins?

> 
> BTW, I made the change to the runtime renaming the function previously 
> known as setCapacity to reserve.  It won't be a property, even if that 
> bug is fixed.
> 
> -Steve

That's a bit of a downer, since a capacity property would have nice 
symmetry with the length property.  I suppose there were good reasons 
though.  Considering the name change, does that mean reserve can only 
reserve new space, i.e. it can't free any that's already been allocated? 
  (That makes me wonder:  Out of curiosity, how does the garbage 
collector know how much space is allocated to a dynamic array or 
especially to a void pointer?  I suppose it's registered somewhere?)