std.concurrency wrapper over MPI?

[jdrewsen]

Den 06-08-2011 05:51, dsimcha skrev:
> I've finally bitten the bullet and learned MPI
> (http://en.wikipedia.org/wiki/Message_passing_interface) for an ultra
> computationally intensive research project I've been working on lately.
> I wrote all the MPI-calling code in D against the C API, using a very
> quick-and-dirty (i.e. not releasable) translation of the parts of the
> header I needed.
>
> I'm halfway-thinking of writing a std.concurrency-like interface on top
> of MPI in D. A few questions:
>
> 1. Is anyone besides me interested in this?
>
> 2. Is anyone already working on something similar.
>
> 3. Would this be Phobos material even though it would depend on MPI, or
> would it better be kept as a 3rd party library?

I think std.concurrency needs to define a new interface for passing 
messages out-of-process ie. other process or host. The implementation 
itself should probably be 3rd party since there are many serialized 
representations and protocols out there to pick from.

> 4. std.concurrency in its current incarnation doesn't allow objects with
> mutable indirection to be passed as messages. This makes sense when
> passing messages between threads in the same address space. However, for
> passing between MPI processes, the object is going to be copied anyhow.
> Should the restriction be kept (for consistency) or removed (because it
> doesn't serve much of a purpose in the MPI context)?
 >
> 5. For passing complex object graphs, serialization would obviously be
> necessary. What's the current state of the art in serialization in D? I
> want something that's efficient and general first and foremost. I really
> don't care about human readability or standards compliance (in other
> words, no XML or JSON or anything like that).

AFAIK David Nadlinger is handling serialization in his GSOC Thrift 
project that he is working on currently.

/Jonas