std.concurrency wrapper over MPI?

[David Nadlinger]

On 8/7/11 12:09 AM, dsimcha wrote:
> On 8/6/2011 5:38 PM, jdrewsen wrote:
>> AFAIK David Nadlinger is handling serialization in his GSOC Thrift
>> project that he is working on currently.
> Good to know, but what flavor?

The most important thing to note, and the reason it could not be 
appropriate for what you want to do, is that the intended main use case 
for Thrift is to define an interface that can easily be used from 
several programming languages, feeling »native« for each of them 
(similar to what protobuf does). As a consequence, Thrift by design only 
supports value types, so it is not possible to e.g. serialize a tree or 
a DAG without »flattening« it first.

Another important feature of Thrift is protocol versioning – you can 
have required and optional fields, and the order of struct fields on the 
wire is not defined. While the schemata themselves are never serialized, 
the serialized data includes type tags and field ids for this purpose.

For the actual serialization format, there are several choices 
available, currently implemented for D are the most popular ones: 
Binary, which basically just dumps the raw bytes to the stream (all 
numbers are written in network byte order, though), Compact, which is a 
space-optimized binary protocol (zigzag varints, merging of some bytes 
where you know you don't need all bits, …), and a »rich« JSON format.

These features obviously come at a (manageable) performance cost, but 
except for that, the code is quite heavily optimized for reading/writing 
performance. For example, while the protocols and transports (serialized 
data sources/sinks) are pluggable at runtime, it is possible to 
specialize all the serialization/RPC code for the actual implementations 
used, thus eliminating all virtual calls and allowing e.g. the 
serialization code for a struct to be inlined into a single function 
without any control flow resp. the reading code into a single switch 
statement (for the field ids) inside a loop.

But as said above, the second item from your list, flexibility with 
regard to the types serialized, is a non-goal for Thrift, so it probably 
isn't the best fit for your application.

David