Orange 1.0.0 beta - serialization library

[Jacob Carlborg]

On 2011-08-20 18:16, dsimcha wrote:
> This looks really great! I'll try it out sometime later. A few questions:
>
> 1. What other archiver formats besides XML might be useful? (I remember
> binary can't work because of the way keys work, though messagepack is
> probably a better option for lightweight serialization in general.
> Orange's niche is portability and serializing as many types as possible.)

I assume any archive type that can unpack a value given a key will work. 
I assume JSON and YAML will work as well. I no nothing about binary 
serialization but I think messagepack can work.

Another problem might be pointers and slices. For example, at the first 
step every array is assumed to be pointing to its own memory, i.e. not a 
slice. After everything is serialized the serializer post process the 
serialized data and replaces all arrays that points into another array 
with a slice.

> 2. What are the prospects for submitting this for inclusion in Phobos?
> Serialization is something basic and universally needed enough that it
> should not require a third-party library.

 From my part:

* Remove all Tango/D1 specific code
* Add documentation
* There are some few things left in the rewrite process
* I've made some modifications to the XML module in Phobos, these need 
to be applied as well
* Probably something else I can't think of right now

I'm not particular happy about removing the Tango/D1 specific code if 
I'm not sure that the library can be included in Phobos.

> 3. Given that all of the necessary introspection already works, what are
> the prospects for creating a Clone archiver? A Clone archiver would not
> actually serialize anything and instead would deep clone whatever data
> structure it receives. If Orange gets into Phobos, cloning could be
> recognized by std.concurrency as a safe, simple way to pass complex
> object graphs between threads. I'm picturing an API something like this:
>
> import std.stdio, whateverOrangeEndsUpBeingNamed.
>
> struct Cloned(T) {
> T obj;
> }
>
> Cloned!T cloned(T)(T obj) { return typeof(return)(obj); }
>
> void spawmMe() {
> receive((int[][]) { writeln("Received an int[][]."); });
> }
>
> void main() {
> auto tid = spawn(&spawnMe);
> auto mat = new int[][](42, 42);
>
> // FAILS: Implicit sharing.
> tid.send(mat);
>
> // Works. mat is automatically deep cloned and the clone is sent
> // to tid. send recognized the Cloned struct as special and
> // processes the payload accordingly.
> tid.send(cloned(mat));
> }

Currently you could serialize the data an pass it send it over to 
another thread and deserialize on the other side, but I'm guessing this 
would be not very effective. I'm not exactly sure what's needed to be 
done to make a clone archive and what can be shared but as you say, the 
necessary introspection already work.

> On 8/20/2011 11:13 AM, Jacob Carlborg wrote:
>> I've almost finished the rewrite of my serialization library Orange. I'm
>> hoping that someone wants to give it a try and see what issues/bugs are
>> found.
>>
>> Project page: http://dsource.org/projects/orange
>> Source code: https://github.com/jacob-carlborg/orange
>>
>> There are two usage examples on the project page. For more examples I
>> recommend looking at the unit tests in the "tests" directory.
>>
>> Description:
>>
>> Orange is a serialization library for D1 and D2, supporting both Tango
>> and Phobos. It can serialize most of the available types in D, including
>> third party types and can serialize through base class references. It
>> supports fully automatic serialization of all supported types and also
>> supports several ways to customize the serialization process. Orange has
>> a separate front end (the serializer) and back end (the archive) making
>> it possible for the user to create new archive types that can be used
>> with the existing serializer.
>>
>> Features:
>>
>> * Automatically serializes the base classes
>> * Supports events (before and after (de)serializing)
>>
>> * Supports non-serialized fields and classes (you can say that some
>> fields in a class should not be serialized)
>>
>> * Licensed under the Boost license
>> * Std/runtime library independent
>>
>> * Extendable - possible to create new archive types and use them with
>> the existing serializer
>>
>> * Serializes through base class references
>> * Serializes third party types
>>
>> * Customization of the (de)serialization process, both intrusive and
>> non-intrusive
>>
>> * Properly (de)serializes slices and pointers
>>
>> Known Issues/Limitations:
>>
>> * Due to limitations in the XML module provided by Phobos the XMLArchive
>> will only work with "char" as the template type with D2
>>
>> * Due to several bugs/limitations in the compiler/runtime even the D2
>> version requires you to register the type when serializing through base
>> class references
>>
>> * No built-in support for versioning
>> * Floating point numbers are not serialized as hexadecimal (D1)
>>
>


-- 
/Jacob Carlborg