[phobos] Interest in having a serializer in Phobos?

[Andrei Alexandrescu]

This exchange involves serialization design. It replies to Michel's 
email on Aug 8.

On 8/8/10 7:26 CDT, Michel Fortin wrote:
> Le 2010-08-08 à 1:47, Andrei Alexandrescu a écrit :
[snip]
>
> My own unreleased, unfinished and in-need-of-a-refactoring
> serialization module does that... but unfortunately dynamically
> recreating the right type cannot be so straightforward in the current
> state of runtime reflection.
>
> This post turned out longer that I expected, please stay with me.

OK. Apologies for taking so long.

> Runtime reflection currently gives you access *only* to the default
> constructor, so this is what my module do internally when
> unserializing a class:
>
> ClassInfo c = findClass(classNameFromSerializationStream); Object o =
> c.create(); (cast(Unserializable)o).unserialize(serialiationStream);

Yes, and that's entirely sensible. You create an empty object using 
standardized header information from the stream, and then you fill it 
with type-specific information by continuing down the stream. It's good 
practice.

> Since we can't access a constructor with a different signature, we
> can't unserialize directly from the constructor.

I think that would be ill-advised too. It's not constructor's job to 
deserialize.

> This is rather a
> weak point as it forces all objects to have a default constructor.

Only all objects that want to support serialization.

> Another options is for the user to manually register his own
> constructor with the serialization system prior unserializing, but
> that's much less convenient.

Agreed.

> The unserialize member function called above must be explicitly added
> by the user (either manually or with a mixin) because the fields
> don't reflect at runtime and the actual class is unknown at
> compile-time. And the class needs to conform to an interface that
> contains that unserialize function so we can find it at runtime.

Yah, good point. Probably we'll need to add to the information emitted 
automatically by the compiler, but for starters, how about this:

class Widget : Serializable!Widget {
    ...
}

Then Serializable would use compile-time introspection to figure out 
Widget's fields etc.

> So before adding a serialization library, I would suggest we solve
> the runtime-reflection problem and find a standard way to attach
> various attributes to types and members.

If we go with the pattern I suggested above, we could even establish a 
naming convention for fields...

> That could be done as a
> library, but ideally it'd have some help from the compiler which
> could put this stuff where it really belongs: ClassInfo. Currently,
> QtD has its own mixins for that, my D/Objective-C bridge has its own
> mixins and class registration system, my serialization module has its
> own, surely Orange has its own, I believe PyD has its own... this is
> going to be a mess pretty soon if it isn't already.

Not necessarily if it's in the standard library and if it facilitates 
everybody else's implementation and higher abstractions.

> Once we have a proper standardized runtime-reflection and attribute
> system, then the serialization module can focus on serialization
> instead of implementing various hacks to add and get to the
> information it needs.

I think we need to start with solid compile-time reflection and then 
look for ways to transport it to runtime, ideally using library facilities.


Andrei