Revamped concurrency API

[Jeremie Pelletier]

Fawzi Mohamed wrote:
> On 2009-10-13 07:45:41 +0200, Andrei Alexandrescu 
> <SeeWebsiteForEmail at erdani.org> said:
> 
>> MIURA Masahiro wrote:
>>> Sean Kelly wrote:
>>>> void sendmsg(T)( Pid pid, T val );
>>>> final void recvmsg(T...)( Pid pid, T ops );
>>>> Pid spawn(T)( T fun );
>>>>
>>>> spawn() is pretty limited so far in that it only spawns threads--I'd
>>>> expect that function to end up with multiple overloads at some point.
>>>
>>> Interesting.  Future spawn() may spawn the thread in a remote CPU that
>>> doesn't share the memory.  In that case it perhaps helps to limit fun to
>>> be pure, and val to be immutable, as in Erlang.  D already has pure and
>>> transitive const, so the first few steps for the massively-parallel
>>> D programs are already completed! :-)
>>
>> That's the plan, except that we weren't thinking of creating remote
>> processes, only sending messages across process and machine boundaries.
>> Here's where notions like value vs. reference and deep cloning become
>> very important.
>>
>>
>> Andrei
> 
> I think that there are different parallelization levels, and different 
> strategies, there isn't ne that "rules them all".
> I am very suspicious of systems where data is moved around "magically", 
> normally the cost of that cannot be ignored, so for coarse level 
> parallelization the message passing approach with explicit data 
> distribution done by the programmer is the way to go.
> On Numa, or really well connected machines having some kind of shared 
> memory is an option, this works well especially if the data is immutable.
> Finally when one really has shared memory one can go to task scheduling 
> without thinking too much about transfer of memory, it is this last 
> thing that I did address in my previous post.
> 
> I think that threads are the wrong approach to parallelization at that 
> level, so what I did was to create Tasks that one can use to express 
> more complex relationships between them, so that one can avoids locks 
> almost always, which is both more efficient and less error prone.

I also don't believe one model is "ruling them all".

I agree that threads aren't the best approach, even things like async 
I/O can be done with futures, other operations can be done with async 
methods, the actor model is perfect for lots of independent operations 
that can be executed in any order, message passing is great for event 
driven designs such as GUIs, software transactional memory works great 
with state management, and those are only a few models.

Threads only need a platform abstraction in the library, from which all 
other concurrency models can be built.

Jeremie