std.concurrency and fibers

[Timon Gehr]

On 10/04/2012 01:32 PM, Alex Rønne Petersen wrote:
> Hi,
>
> We currently have std.concurrency as a message-passing mechanism. We
> encourage people to use it instead of OS threads, which is great.
> However, what is *not* great is that spawned tasks correspond 1:1 to OS
> threads. This is not even remotely scalable for Erlang-style
> concurrency. There's a fairly simple way to fix that: Fibers.
>
> The only problem with adding fiber support to std.concurrency is that
> the interface is just not flexible enough. The current interface is
> completely and entirely tied to the notion of threads (contrary to what
> its module description says).
>
> Now, I see a number of ways we can fix this:
>
> A) We completely get rid of the notion of threads and instead simply
> speak of 'tasks'. This trivially allows us to use threads, fibers,
> whatever to back the module. I personally think this is the best way to
> build a message-passing abstraction because it gives enough transparency
> to *actually* distribute tasks across machines without things breaking.
> B) We make the module capable of backing tasks with both threads and
> fibers, and expose an interface that allows the user to choose what kind
> of task is spawned. I'm *not* convinced this is a good approach because
> it's extremely error-prone (imagine doing a thread-based receive inside
> a fiber-based task!).
> C) We just swap out threads with fibers and document that the module
> uses fibers. See my comments in A for why I'm not sure this is a good idea.
>
> All of these are going to break code in one way or another - that's
> unavoidable. But we really need to make std.concurrency grow up; other
> languages (Erlang, Rust, Go, ...) have had micro-threads (in some form)
> for years, and if we want D to be seriously usable for large-scale
> concurrency, we need to have them too.
>
> Thoughts? Other ideas?
>

+1, but what about TLS?