TempAlloc (Was: Library Development: What to finish/flesh out?)

[Masahiro Nakagawa]

On Sun, 27 Mar 2011 12:09:03 +0900, dsimcha <dsimcha at yahoo.com> wrote:

> On 3/18/2011 2:56 AM, Don wrote:
>>> 3. TempAlloc: A memory allocator based on a thread-local segmented  
>>> stack,
>>> useful for allocating large temporary buffers in things like numerics
>>> code.
>>> Also comes with a hash table, hash set and AVL tree optimized for this
>>> allocation scheme. The advantages over plain old stack allocation are
>>> that
>>> it's independent of function calls (meaning you can return pointers to
>>> TempAlloc-allocated memory from a function, etc.) and it's segmented,
>>> meaning
>>> you can allocate huge buffers w/o risking stack overflow. Its main
>>> weakness
>>> is that this stack is not scanned by the GC, meaning that you can't
>>> store the
>>> only reference to a GC-allocated piece of memory here. However, in
>>> practice
>>> large arrays of primitives are an extremely common case in
>>> performance-critical code. I find this module immensely useful in
>>> dstats and
>>> Lars Kyllingstad uses it in SciD. Getting it into Phobos would make it
>>> easy
>>> for other scientific/numerics code to use it. Completion state:
>>> Working and
>>> used. Needs a litte cleanup and documentation. (Phobos candidate)
>>
>> This is #1. Far and away. Belongs in druntime.
>> I would use it instantly in BigInt.
>
> This is not up for "official" review yet because std.net.isemail and  
> std.parallelism (and possibly more stuff depending on how we manage the  
> review queue) are ahead of it.  If you want a sneak preview:
>
> Code:
>
> https://github.com/dsimcha/TempAlloc
>
> Docs:
>
> http://cis.jhu.edu/~dsimcha/d/phobos/core_tempalloc.html
>
> I put in a few small, tangentially related memory allocation functions  
> that I've wanted in druntime/Phobos for awhile in this proposal, too.  
> alignedMalloc() and alignedFree() are used by TempAlloc but can be made  
> private if people don't like them.  newVoid() is not used by TempAlloc  
> and might fit better in std.array, but I think it belongs somewhere in  
> Phobos or druntime.

Now, I am thinking of light-weight process for concurrency.
Your TempAlloc(segmented stack) seems to be a good parts for implementing  
such feature :)


Masahiro