std.allocator needs your help

[Dan Schatzberg]

On Tuesday, 24 September 2013 at 16:06:39 UTC, Andrei 
Alexandrescu wrote:
> On 9/24/13 4:38 AM, Dan Schatzberg wrote:
>> One thing I'm not sure is addressed by this design is memory 
>> locality. I
>> know of libnuma http://linux.die.net/man/3/numa which allows 
>> me to
>> express what NUMA domain my memory should be allocated from at 
>> run-time
>> for each allocation.
>>
>> In the case that I want to allocate memory in a specific NUMA 
>> domain
>> (not just local vs non-local), I believe this design is 
>> insufficient
>> because the number of domains are only known at run-time.
>>
>> Also, as far as alignment is concerned I will throw in that 
>> x86 is
>> relatively unique in having a statically known cache-line 
>> size. Both ARM
>> and PowerPC cores can differ in their cache-line sizes. I feel 
>> this is a
>> significant argument for the ability to dynamically express 
>> alignment.
>
> Could you send a few links so I can take a look?
>
> My knee-jerk reaction to this is that NUMA allocators would 
> provide their own additional primitives and not participate 
> naively in compositions with other allocators.
>
>
> Andrei

Not sure what kind of links you're looking for

The following link is a good discussion of the issue and the 
current solutions
http://queue.acm.org/detail.cfm?id=2513149

In particular:

"The application may want fine-grained control of how the 
operating system handles allocation for each of its memory 
segments. For that purpose, system calls exist that allow the 
application to specify which memory region should use which 
policies for memory allocations.

The main performance issues typically involve large structures 
that are accessed frequently by the threads of the application 
from all memory nodes and that often contain information that 
needs to be shared among all threads. These are best placed using 
interleaving so that the objects are distributed over all 
available nodes."

The Linux/libc interfaces are linked in my first comment. 
Specifically with the mbind() call one can specify the policy for 
allocations from a virtual address range (which NUMA node to 
allocate the backing physical page from). More generally you 
could imagine specifying this per allocation.

What is your objective though? Aren't you trying to define a 
hierarchy of allocators where more specific allocators can be 
composed from general ones? In which case what is the concern 
with including locality at the base level? It seems to be one 
characteristic of memory that programmers might be concerned with 
and rather trivially you can compose a non-locality aware 
allocator from a locality aware allocator.