Lock-Free Actor-Based Flow Programming in D2 for GSOC2011?

[nonexistent]

Piotr Szturmaj Wrote:

> eris wrote:
> > I used the Tango Fibers implementation (thanks Sean Kelly I believe) and various
> > reactor libraries to implement the actor engine.
> 
> I'm working on something similar, e.g. event-driven programming using 
> fibers. I need it for my upcoming network library.
> 
> But I don't use queue for each fiber, but instead each fiber "poll" for 
> events so there's no possibility of queue overflow.

Why not implement cross thread messaging and then use task stealing. Then you could set up a couple of default threads and add maybe up to 10 more on top as decidedly needed and each thread accesses a list (it's own list) of fibers to be executed. When one thread runs out of fibers to execute, instead of sitting and waiting for something else to get mapped to it, it just goes ahead and steals a fiber from the thread in front of it (which if in turn is empty, steals 2 from the one in front so it can have one and the other as well). You could of course change the granularity of steals so as to keep each thread with enough tasks. The only thing is that each of these threads would be a "StealThread" but something would have to keep a list of them and map events dynamically to each "StealThread" in the list. Instead of you being able to map to each thread individually, the list holder would do that for you in a way that tries to keep them full. Now, since each thing in the program could actually just be a fiber to be ran by the "StealThread", you could just map one function to a "StealThread" that would recursively map itself and other functions back into the Threads so as to continue execution of the program, you'd have a program that is multithreaded and pretty much runs on it's own without making actuall calls besides map this or that to a thread. This could turn out to be very flexible (especially if you could find a way to map as many "StealThread" functions into fibers to be passed to a "StealThread"). This is a very good method for servers because you don't want any thread to be still just because no tasks are mapped to it. The effeciency comes from stealing tasks, keeping all the little threads going.