std.concurrency and fibers

[Sean Kelly]

On Oct 4, 2012, at 4:32 AM, Alex Rønne Petersen <alex at lycus.org> wrote:

> Hi,
> 
> We currently have std.concurrency as a message-passing mechanism. We encourage people to use it instead of OS threads, which is great. However, what is *not* great is that spawned tasks correspond 1:1 to OS threads. This is not even remotely scalable for Erlang-style concurrency. There's a fairly simple way to fix that: Fibers.
> 
> The only problem with adding fiber support to std.concurrency is that the interface is just not flexible enough. The current interface is completely and entirely tied to the notion of threads (contrary to what its module description says).

How is the interface tied to the notion of threads?  I had hoped to design it with the underlying concurrency mechanism completely abstracted.  The most significant reason that fibers aren't used behind the scenes today is because the default storage class of static data is thread-local, and this would really have to be made fiber-local.  I'm reasonably certain this could be done and have considered going so far as to make the main thread in D a fiber, but the implementation is definitely non-trivial and will probably be slower than the built-in TLS mechanism as well.  So consider the current std.concurrency implementation to be a prototype.  I'd also like to add interprocess messaging, but that will be another big task.