std.concurrency and fibers

Sean Kelly sean at invisibleduck.org
Fri Oct 5 07:26:08 PDT 2012


On Oct 4, 2012, at 9:18 PM, Alex Rønne Petersen <alex at lycus.org> wrote:

> On 05-10-2012 01:30, Sean Kelly wrote:
>> On Oct 4, 2012, at 4:32 AM, Alex Rønne Petersen <alex at lycus.org> wrote:
>> 
>>> Hi,
>>> 
>>> We currently have std.concurrency as a message-passing mechanism. We encourage people to use it instead of OS threads, which is great. However, what is *not* great is that spawned tasks correspond 1:1 to OS threads. This is not even remotely scalable for Erlang-style concurrency. There's a fairly simple way to fix that: Fibers.
>>> 
>>> The only problem with adding fiber support to std.concurrency is that the interface is just not flexible enough. The current interface is completely and entirely tied to the notion of threads (contrary to what its module description says).
>> 
>> How is the interface tied to the notion of threads?  I had hoped to design it with the underlying concurrency mechanism completely abstracted.  The most significant reason that fibers aren't used behind the scenes today is because the default storage class of static data is thread-local, and this would really have to be made fiber-local.  I'm reasonably certain this could be done and have considered going so far as to make the main thread in D a fiber, but the implementation is definitely non-trivial and will probably be slower than the built-in TLS mechanism as well.  So consider the current std.concurrency implementation to be a prototype.  I'd also like to add interprocess messaging, but that will be another big task.
> 
> Mostly in that everything operates on Tids (as opposed to some opaque Cid type) and, as you mentioned, TLS. The problem is basically that people have gotten used to std.concurrency always using OS threads due to subtle things like that from day one.

A Tid is a Cid and in the first iteration I actually named it Cid and was asked to change it.  Tid seems reasonable since it represents a logical thread anyway. It just may not actually be a kernel thread. I think we have to make TLS work for fibers or using them isn't an option. It would be ridiculous to say "D has this cool new idea about statics but you can't use it if you're using the standard concurrency package."


More information about the Digitalmars-d mailing list