Lock-Free Actor-Based Flow Programming in D2 for GSOC2011?

[eris]

Sorry for the hasty overview, I was trying to squeeze my request in under the wire
(and failed).

My previous system (Dendrite) was a D1/Tango implementation of flow-based
programming environment similar to the BBC Kamaelia/Axon system implemented in
Python.  It was moderately challenging and fun, but really hard to avoid the
Turing Tarpit of a universal programming system.

I used the Tango Fibers implementation (thanks Sean Kelly I believe) and various
reactor libraries to implement the actor engine.

After I finished 0.9, one of the chat members (mwarning maybe?) asked me if it was
multicore. (It wasn't).  This seemed like such an obvious and straight forward
addition that (like a fool) I set about implementing it.  (Tarpit 2)

Last time I finished something on that branch it was a high-speed lock-free
message FIFO so the Fiber Cores could communicate easily.  I achieved about 50-60
MT/s and figured I needn't go any further on that. I also integrated the portable
hwloc library so I could introspect the runtime multicore topology and set the
Thread affinity.

In any case, the system basically allows you to define pipe-like (and simple
mailbox msg transfer) systems that process data by having concurrent actors
reading from their inputs, processing them and writing to their outputs.  Fun for
the whole family.

So this is nothing tremendously new, flow-based actor systems are implemented in a
number of languages and libraries.  Higher level concurrency systems like LINDA
can be built from actor systems etc.

After I came up for air after implementing the multicore flow-based components D2
had really come into it's own.  I read TDPL (yea!) with special attention to the
concurrency system and I'm really excited to see how I can improve on my Dendrite
system using D2-supported constructs (I know Fiber is in Phobos2 so that's good).
 Elegant mailbox support and lock-free patterns are built right into the
concurrency system so that makes my life a lot easier if I can use them.  (Thanks
Andrei et al)

Right now I'm trying to port to ldc2/phobos2.  As soon as I get a working build
things will go a lot better.

That's basically the idea.  There's a lot more that I haven't mentioned, like the
fact that the reactor, scheduler and other engines are starting to take on aspects
of an operating system or virtual machine.  Am I re-inventing Mach? Poorly?

That's the "overview".

eris




So, I started working back up my list of outstanding multicore patches