std.concurrency wrapper over MPI?

[Sean Kelly]

I'd love to be able to send classes between processes, but first we need a good serialization/deserialization mechanism. 

Sent from my iPhone

On Aug 5, 2011, at 8:51 PM, dsimcha <dsimcha at yahoo.com> wrote:

> I've finally bitten the bullet and learned MPI (http://en.wikipedia.org/wiki/Message_passing_interface) for an ultra computationally intensive research project I've been working on lately.  I wrote all the MPI-calling code in D against the C API, using a very quick-and-dirty (i.e. not releasable) translation of the parts of the header I needed.
> 
> I'm halfway-thinking of writing a std.concurrency-like interface on top of MPI in D.  A few questions:
> 
> 1.  Is anyone besides me interested in this?
> 
> 2.  Is anyone already working on something similar.
> 
> 3.  Would this be Phobos material even though it would depend on MPI, or would it better be kept as a 3rd party library?
> 
> 4.  std.concurrency in its current incarnation doesn't allow objects with mutable indirection to be passed as messages.   This makes sense when passing messages between threads in the same address space. However, for passing between MPI processes, the object is going to be copied anyhow.  Should the restriction be kept (for consistency) or removed (because it doesn't serve much of a purpose in the MPI context)?
> 
> 5.  For passing complex object graphs, serialization would obviously be necessary.  What's the current state of the art in serialization in D? I want something that's efficient and general first and foremost.  I really don't care about human readability or standards compliance (in other words, no XML or JSON or anything like that).