Variable-length stack allocated arrays

[Lars T. Kyllingstad]

dsimcha wrote:
> == Quote from Chad J (chadjoan at __spam.is.bad__gmail.com)'s article
>> http://d.puremagic.com/issues/show_bug.cgi?id=3696
>> I'm realizing now that returning an object allocated on this stack is a
>> tricky task unless there is help from the caller.  The lifetime of the
>> object has to be determined somehow.  The Tango convention makes it
>> fairly explicit that the parent's scope will be the beginning and the
>> end for the memory.  If it needs to live longer then the caller just
>> passes a buffer that isn't stack/scoped memory.  Only the caller really
>> knows how long the memory needs to hang around.  hmmmmm.
> 
> The optional buffer idiom is great for stuff that's returned from a function.  I
> use it all the time.  On the other hand, for temporary buffers that are used
> internally, whose mere existence is an implementation detail, having the caller
> maintain and pass in a buffer is a horrible violation of encapsulation and good
> API design, even by the standards of performance-oriented APIs.  Even if you know
> it's never going to change, it's still annoying for the caller to even have to
> think about these kinds of details.

The design I'm using for SciD is just that,

   double[] algo(params, double[] buffer=null, double[] workspace=null);

and it's already starting to annoy me.  I never remember how much 
workspace memory is required for each algorithm, and constantly have to 
look it up.  I definitely have to do something about it.

(The worst are the LAPACK linear algebra functions. There you have a 
minimum workspace size, which is easily found in the docs, but for many 
of them you also have an *optimal* workspace size, which you have to 
make LAPACK calculate for you.  And even the minimum size depends on a 
lot of factors, like matrix size (naturally), whether the matrix is real 
or complex, symmetric or hermitian, triangular, etc.)


>> Still that SuperStack thing would be useful.  It's like alloca but
>> better (look ma, no cast) and slightly more general (ex: it can be freed
>> in parent's scope, if you are willing to play it risky).
> 
> Yeah, for about the past year I've been playing around with TempAlloc, wich was
> basically me taking Andrei's SuperStack idea when it was first proposed and
> running with it because I needed it sooner rather than later.  It's basically
> encapsulated in dstats.alloc
> (http://svn.dsource.org/projects/dstats/docs/alloc.html).

TempAlloc looks really interesting.  I was actually thinking about 
writing some kind of "preallocated memory pool" mechanism myself, to get 
rid of all the workspace[]s. If you don't mind, I'll look into using 
TempAlloc for SciD.


> [...]
> It's got the following advantages over alloca:
> 
> 1.  Can't overflow unless you're completely out of address space.  As a last
> resort, it allocates another chunk of memory from the heap.

When all chunks in a block have been TempAlloc.free()'ed, is the block 
GC.free()'ed as well?  Or is the memory retained for future use?  (And 
if the answer to the last question is 'no', is there a reason not to 
retain the memory until the program exits, or at least to include an 
option to?)

-Lars