Distributed Memory implementation

tcak via Digitalmars-d digitalmars-d at puremagic.com
Wed Jan 20 00:21:11 PST 2016


On Wednesday, 20 January 2016 at 07:58:20 UTC, Marco Leise wrote:
> Am Mon, 18 Jan 2016 05:59:15 +0000
> schrieb tcak <1ltkrs+3wyh1ow7kzn1k at sharklasers.com>:
>
>> I, due to a need, will start implementation of distributed 
>> memory system.
>> 
>> Idea is that:
>> 
>> Let's say you have allocated 1 GiB space in memory. This 
>> memory is blocked into 4 KiB.
>> 
>> After some reservation, and free operations, now only the 
>> blocks 0, 12, and 13 are free to be allocated.
>> 
>> Problem is that those memory blocks are not consecutive.
>
> Well ... there is a way that is a bit arcane. If you don't
> need a lot of such remappings and you use page sized blocks
> anyways, you could remap the scattered virtual memory to a new
> consecutive range and be done.
> Basically you reserve a new uncommitted (e.g. not backed by
> physical memory) virtual memory range, remap your scattered
> blocks into that range in the desired order and then release
> the original blocks. I used this technique in a circular
> buffer implementation, mirroring the start of the buffer to the
> end, thus avoiding the wrapping problems that normally occur
> when accessing data close to the end in a traditional
> circular buffer.
> Caveats: You'd need to allocate/free virtual memory directly
> in multiples of the allocation granularity (which is the same
> as the page size on POSIX) and you put some stress on the
> virtual memory system. A process may have at most 2^16
> memory mappings at a time on my Linux system.

What you are saying is true, and before starting the 
implementation, it was
in my mind, but it doesn't fit to shared memory structure. If 
very big part
of memory is already used, there won't be any more space to 
allocate a
consecutive memory to bring the content of non-consecutive memory 
blocks.

Anyway, I am done with the implementation. At least implemented 
as "shared" class.
I am aware that, because of calculations, it won't be as fast as 
linear memory
access, but solve the problem at least. I am writing unittest 
right now.


More information about the Digitalmars-d mailing list