Safe & Performant Inter-Thread Message Passing

[deadalnix]

On Tuesday, 18 February 2014 at 21:37:09 UTC, Nordlöw wrote:
> On Tuesday, 18 February 2014 at 21:21:54 UTC, deadalnix wrote:
>> On Tuesday, 18 February 2014 at 21:05:38 UTC, Nordlöw wrote:
>>> What's the best solution to communicate between threads in D 
>>> today if I care about
>>>
>>> 1. Security & Correctness?
>>> 2. Performance?
>>>
>>> and are these mutually exclusive?
>>>
>>> Does Phobos or other library contain lockfree queues?
>>>
>>> From what I can see std.concurrent.MessageBox requires 
>>> Mutex-locking. Is this really needed or can it be enhanced to 
>>> support lockfree communication similar to what Boost.Lockfree 
>>> does?
>>
>> In its current implementation, it uses mutexes (or did last 
>> time
>> I checked). However, this isn't required.
>>
>> I wanted to implement something a while ago, but was stopped by
>> the fact that DMD doesn't implement shared as per spec (ensure
>> sequential consistency). It is still possible to do with
>> primitives for concurrency available in the runtime.
>>
>> If you plan to submit a pull request, I'll be happy to review.
>
> That is good to know.
>
> I have no experience in writing threadlocking queues.
> I'm just aware of what's state of the art.
>
> It would be interesting to take look though... :)
>
> A few thoughts and questions:
> - I do know that you construct it using CPU intrinsics like CAS 
> and
> that single producer single consumer queues is the easiest one 
> to get right.
> - This should suffice for the needs of MessageBox right?
> - I guess we need to make copies of all the messages right?
>
> Thx

MessageBox can have several producer but simply one consumer. I
had something similar to disruptor in mind :
http://www.slideshare.net/trishagee/introduction-to-the-disruptor

We don't need to make copies. Actually, std.concurency accept
only values types (that will be copied as they are value types),
immutable (that do not need copy as they are immutable) and
shared (where it is up to the programmer to ensure correctness).