asynchronous communication between threads

Charles Hixson charleshixsn at earthlink.net
Thu Jan 3 10:38:43 PST 2013


On 01/03/2013 08:40 AM, Dmitry Olshansky wrote:
> 02-Jan-2013 03:54, Charles Hixson пишет:
>> If I were to use the below as an asynchronous communication channel,
>> would it avoid deadlocks (presuming that only Cell called Msg) and that
>> when a thread activated Cell, the first thing it did was process it's
>> mailbox?
>> Also, if only around 7 cells were created in the main thread, would RAM
>> usage remain reasonable (i.e., when a thread was through with an MCell,
>> would only one copy continue to exist in RAM? Or would there be a copy
>> for each thread that had ever referenced it? Should MCell instances be
>> marked shared?
>
> That's a lot of questions to ask and currently I hardly can decipher the
> whole logic from this and the code below alone, too much is behind the
> scenes.
>
> I'm willing to help out if you could post a more complete example or
> more accurate questions as e.g. "created 7 cells in main thread" could
> be done very differently.
>
> Also a nit at this size of code (and for me it's unformatted) I'd
> recommend linking to the code on some paste service e.g. dpaste
> http://dpaste.dzfl.pl/.
>
> [snip]
>
I'm trying to design a program (I'm not writing it yet) that requires 
asynchronous communication between LOTS of separate "cells".  You can 
think of it as vaguely like a neural network, but it doesn't fit the 
definition, that's just an analogy.  I thought that D would be a good 
language to do this in, but all the built-in approaches seem to be 
"overly protective", locking a lot more than I want and yielding 
deadlocks, or copying more than I want, and ensuring that I end up with 
stale data.  The approach that I've come up with to avoiding this is to 
attach a mailbox to each "cell".  In an ideal world, each cell would be 
a separate thread, but as there will be at least hundreds of thousands 
of them, and likely millions, that's unreasonable.  So I need the 
threads to cycle through pools of "cells".  Each cell may send messages 
to any other cell (not really, but you don't know in advance what the 
connections will be).

That skeleton of code was intended to show the idea of cells isolated 
from outside attached to a mailbox, which has blocking access from the 
outside.  The cells should never block, but this is largely because they 
are only directly accessed by the thread within which they run.  The 
mailboxes can receive messages from anyone, but their processes are so 
short, that blocking will be extremely brief.

I wanted to know if this proposed design would work, as in not getting 
into deadlocks, not blocking excessively, and not giving me excessively 
stale data.  More details aren't available, because I didn't want to 
commit to this design if the basic design was wrong, so I haven't 
written them.  It has been suggested that since the cell will only be 
accessed by the thread, it doesn't need to be synchronous.

I'm really nervous about how many copies of the cell will exist, 
however.  Since there are going to be so many of them, if I ended up 
with a cell/thread, the system would bog down in unused storage.  But 
the mailbox needs to be globally accessible for the scheme to work.

N.B.:  When a cell receives a message from another cell it's likely, but 
not guaranteed, to send a response back.  It may also send responses 
onwards.  And the number of cells isn't fixed, nor is the number of 
their connections.  (This is less important in D than in many other 
languages, but even in D it affects serialization.)

FWIW, I've been told that an approximately similar design has worked in 
Ada, though the design was written in Ada-83.  (In Ada the mailbox was 
protected, which I take to be approximately the same as synchronized.)


More information about the Digitalmars-d-learn mailing list