std.concurrency wrapper over MPI?

[Sean Kelly]

I'm hoping to simply extend the existing API. The crucial portion will be the addition of a Node (base) type. 

Sent from my iPhone

On Aug 6, 2011, at 2:38 PM, jdrewsen <jdrewsen at nospam.com> wrote:

> Den 06-08-2011 05:51, dsimcha skrev:
>> I've finally bitten the bullet and learned MPI
>> (http://en.wikipedia.org/wiki/Message_passing_interface) for an ultra
>> computationally intensive research project I've been working on lately.
>> I wrote all the MPI-calling code in D against the C API, using a very
>> quick-and-dirty (i.e. not releasable) translation of the parts of the
>> header I needed.
>> 
>> I'm halfway-thinking of writing a std.concurrency-like interface on top
>> of MPI in D. A few questions:
>> 
>> 1. Is anyone besides me interested in this?
>> 
>> 2. Is anyone already working on something similar.
>> 
>> 3. Would this be Phobos material even though it would depend on MPI, or
>> would it better be kept as a 3rd party library?
> 
> I think std.concurrency needs to define a new interface for passing messages out-of-process ie. other process or host. The implementation itself should probably be 3rd party since there are many serialized representations and protocols out there to pick from.
> 
>> 4. std.concurrency in its current incarnation doesn't allow objects with
>> mutable indirection to be passed as messages. This makes sense when
>> passing messages between threads in the same address space. However, for
>> passing between MPI processes, the object is going to be copied anyhow.
>> Should the restriction be kept (for consistency) or removed (because it
>> doesn't serve much of a purpose in the MPI context)?
> >
>> 5. For passing complex object graphs, serialization would obviously be
>> necessary. What's the current state of the art in serialization in D? I
>> want something that's efficient and general first and foremost. I really
>> don't care about human readability or standards compliance (in other
>> words, no XML or JSON or anything like that).
> 
> AFAIK David Nadlinger is handling serialization in his GSOC Thrift project that he is working on currently.
> 
> /Jonas