What would you do... -> Extending std.concurrency

[Andrew Wiley]

On Thu, Dec 8, 2011 at 7:12 PM, Sean Kelly <sean at invisibleduck.org> wrote:
> On Dec 8, 2011, at 4:13 PM, Andrew Wiley wrote:
>>
>> This is somewhat of a threadjack, but I was looking at making a
>> lighter actor-like abstraction for message passing, and possibly
>> making it compatible with std.concurrency, and I saw this:
>> "The general idea is that every messageable entity is represented by a
>> common handle type (called a Cid in this implementation), which allows
>> messages to be sent to in-process threads, on-host processes, and
>> foreign-host processes using the same interface."
>>
>> As far as I can tell, Cid does not appear outside the documentation at
>> the top of std.concurrency, and Tid is a direct linkage to
>> std.concurrency's private mailbox implementation, so I was wondering
>> what the best way to extend message-passing functionality to another
>> type would be.
>
> Originally I'd thought that Tid would be process-local and we'd have a different type for external stuff, but that sacrifices a lot of the power of the actor model.  However, I think everything can be handled by the one Tid type.  The real trick is meshing the template-driven send() operation with different transport (and serialization) schemes, but I've become convinced that this can work.  At some point a Node type will be added, and Nodes will have a transport interface.  So calling send() on a Tid that contains a Node reference will serialize to the transport layer.  If the Node reference is null then it will work as it does now.  So what you'll ultimately want to do is provide a transport mechanism for the method you desire.

That would mean adding a reference to the Tid for every type we would
want to be able to send messages to. If you just look at Node types
and remote actors, that's not so bad, but what I'm looking towards is
lighter local actors, IE fiber based or closure based. The problem
with thread based actors is that you can really only spin up a few
thousand of them. Fibers get maybe an order of magnitude farther, but
closures can get into the millions by sacrificing stack-based state.
If anyone wants to try to implement more fine-grained concurrency
using the actor model, this quickly becomes necessary (heck, it's one
of the mainstays of Scala's standard library and the Akka library).


I think Tid is basically a wrapper around a MessageBox, and the
abstraction we actually want is much closer to MessageBox's public
API, something like:
interface Actor { void send(Message msg); }
The Message should probably also store some way of replying to it when
applicable, and there might be a way to get a Future of some sort when
sending a message so you could wait for the reply.
(And there would be a convenient templated way to make arbitrary types
into Messages - probably just a method in the Actor base class)

Actors are going to involve heap allocation anyway, and even with
remote actors, you'll probably wind up with some sort of proxy object
on the local system representing an actor or a set of actors somewhere
else, and it seems more natural to me to directly pass references
around and let polymorphism handle actually getting the message to its
destination.

One other detail that would need to change to support remote actors is
that right now you check hasLocalAliasing, which allows shared data.
If we're going to support remote actors, it needs to be hasAliasing
(which is even more fun because Variant currently can't handle
immutable types), and if you want to continue to support sending
shared data locally, we have to define a notion of a LocalMessage as
well as Message (where LocalMessage would still use hasLocalAliasing
and Message could implicitly convert to LocalMessage).