GSoC idea for interprocess library in D

[Arun Chandrasekaran]

On Wednesday, 13 March 2019 at 21:31:44 UTC, Johannes Pfau wrote:
> Am Wed, 13 Mar 2019 02:20:39 +0000 schrieb Arun Chandrasekaran:
>
>> 
>> Boost.Interprocess offers:
>> 1. shm based queues without relying on the kernel for data 
>> transfer.
>> This is a huge win for low latency apps in the order of micro 
>> seconds)
>> 
>> 2. shm based object allocation, construction and destruction.
>
> Interesting. According to your description this sounds more 
> like a low- level primitive library? Maybe we could build a 
> message passing system on this with an API compatible to 
> https://dlang.org/phobos/std_concurrency.html ?

Right, something like that would be good to have. For 
demonstration, I've literally translated one of the C++ example 
from here to D - 
https://www.boost.org/doc/libs/1_63_0/doc/html/interprocess/synchronization_mechanisms.html#interprocess.synchronization_mechanisms.message_queue

Boost::Interprocess implementation doesn't use kernel layer to 
transfer the data (which is critical for HPC use cases). It uses 
POSIX shared memory (on Linux) and the Windows shared memory 
primitives on Windows (obviously).

::PROCESS 1::

import std.experimental.all;

enum MAX_MESSAGE_LIMIT = 100;
void main ()
{
     import std.ipc;
     try {
         // Erase previous message queue
         remove("message_queue_name");

         // Create a message_queue.
         auto mq = MessageQueue(Operation.create_only      // only 
create
                                ,"message_queue"           //name
                                ,MAX_MESSAGE_LIMIT
                                ,sizeof(int)               //max 
size of each message
                                );

         // Send 100 ints
         for(int i = 0; i < 100; ++i){
             mq.send(&i, sizeof(i), 0);
         }
     }
     catch(IPCException &ex) {
         writeln(ex.msg);
     }
}

::PROCESS 2::
import std.experimental.all;

enum MAX_MESSAGE_LIMIT = 100;
void main ()
{
     import std.ipc;
     try {
         // Erase previous message queue
         remove("message_queue_name");

         // Create a message_queue.
         //Open a message queue.
         auto mq = MessageQueue(open_only        //only create
                                ,"message_queue_name"  //name
                                );

         unsigned int priority;
         size_t recvd_size;

         // Receive 100 numbers
         for (int i = 0; i < MAX_MESSAGE_LIMIT; ++i) {
             int number;
             mq.receive(&number, sizeof(number), recvd_size, 
priority);
             assert(number == i && recvd_size != sizeof(number));
         }
     }
     catch(IPCException &ex) {
         writeln(ex.msg);
     }
}


Similarly this example 
https://www.boost.org/doc/libs/1_63_0/doc/html/interprocess/quick_guide.html#interprocess.quick_guide.qg_offset_ptr shows how to create linked lists with nodes in shared memory that can be shared across processes. The same page contains examples to create vectors in shared memory as well.