[OT] Do you see a problem in this serialization format ?

Basile B. via Digitalmars-d digitalmars-d at puremagic.com
Tue Jun 7 07:17:10 PDT 2016


On Tuesday, 7 June 2016 at 11:52:55 UTC, yawniek wrote:
> On Tuesday, 7 June 2016 at 11:22:55 UTC, Basile B. wrote:
>> On Tuesday, 7 June 2016 at 10:24:56 UTC, WebFreak001 wrote:
>>> On Tuesday, 7 June 2016 at 02:11:55 UTC, Basile B. wrote:
>>>> I had to make a custom format because
>> SDL would have the same problem as JSON. I need to store the 
>> type, so key = value does not work and an object (that 
>> contains name = "thename", type = "thetype", value = "value") 
>> must be created for each property. The serialisation data 
>> become huge and is painfull to read.
>
> not necessarily and i don't completely understand why you want 
> to include the schema in each message. this is a lot of bloat. 
> and once you need more complicated datastructures it will be 
> hard anyway and you need manual glue code. e.g. `Dlist!string[] 
> mylist = <here be dragons>`
>
> i would go for protobufs, thrift or if you want something easy 
> to parse then json maybe with json schema.
> plug: try https://github.com/tamediadigital/asdf
> it was more or leass created for what you want.
> - no object instantiation overhead
> - instead of "factory" methods you just use the deserializer 
> methods

The whole schema (so adding the type) is necessary for example to 
store the events targets:

- in a object: @Get bool delegate(string) propertyA();
- in another object bool target(string);
- the target is stored with its type: 
storeRef!(typeof(propertyA))("obj.target", &obj.target)
- propertyA = &obj.target;

The serializer is able to detect that propertyA is assigned and 
that the target is a reference that's stored so it writes

     bool-delegate(string) propertyA = "obj.target"

When deserializing, the serializer will try to find "obj.target" 
in the reference storage and is able to assign the event. If the 
type is not stored then

- target of events can be serialized
- class instances can't be created with a factory
- target of properties that returns an object can't be serialized
- objects that are serialized can't involve:

1 march 2016:

class Segment {}

class Foo
{
   @SetGet Segment segment;
}

written

     Segment segment = ""
     {...}

12 august 2016, for a reason or another the declarations change

class Foo
{
   @SetGet float[] segment;
}

When reloading what's written with the 1/03/16 version in the 
12/08/16 version the serializer detects the type change and an 
event can be used to handle the reloading of the old data. This 
is why the type is included.

Actually the serializer already exists:
https://github.com/BBasile/iz/blob/master/import/iz/serializer.d

But as it will be used in a gui library:
https://github.com/BBasile/kheops

I want to be sure that the format is coherant. Those who know 
Delphi for example knows that object streaming is a master piece. 
Without object streaming nothing works and all the GUI 
construction must be hardcoded.

In addition this library will use this system to apply the style 
of a control (for example the code posted this morning is 
obtained by serializing one of the primitive used to stylize a 
control). If I want to store the type, in JSON I obtain:

{
   "name": "root",
   "type": "Object",
   "value": [
     {
       "name": "alignment",
       "type": "Alignment",
       "value": "none"
     },
     {
       "name": "width",
       "type": "float",
       "value": "200"
     },
     {
       "name": "height",
       "type": "float",
       "value": "75"
     },
     {
       "name": "left",
       "type": "float",
       "value": "10"
     },
     {
       "name": "top",
       "type": "float",
       "value": "180"
     },
     {
       "name": "visible",
       "type": "bool",
       "value": "true"
     },
     {
       "name": "focusable",
       "type": "bool",
       "value": "true"
     },
     {
       "name": "wantMouse",
       "type": "bool",
       "value": "true"
     },
     {
       "name": "caption",
       "type": "string",
       "value": "Reset pos using iz.serializer"
     },
     {
       "name": "fill",
       "type": "Brush",
       "value": [
         {
           "name": "fillKind",
           "type": "FillKind",
           "value": "uniform"
         },
         {
           "name": "color",
           "type": "uint",
           "value": "4294965965"
         },
         {
           "name": "gradientKind",
           "type": "GradientKind",
           "value": "horizontal"
         },
         {
           "name": "fillAdjustment",
           "type": "FillAdjustment",
           "value": "none"
         },
         {
           "name": "gradient",
           "type": "Gradient",
           "value": "Gradient"
         },
         {
           "name": "bitmap",
           "type": "Bitmap",
           "value": [
             {
               "name": "data",
               "type": "Stream",
               "value": "[]"
             }
           ]
         }
       ]
     },
     {
       "name": "stroke",
       "type": "Brush",
       "value": [
         {
           "name": "fillKind",
           "type": "FillKind",
           "value": "uniform"
         },
         {
           "name": "color",
           "type": "uint",
           "value": "4287245282"
         },
         {
           "name": "gradientKind",
           "type": "GradientKind",
           "value": "horizontal"
         },
         {
           "name": "fillAdjustment",
           "type": "FillAdjustment",
           "value": "none"
         },
         {
           "name": "gradient",
           "type": "Gradient",
           "value": "Gradient"
         },
         {
           "name": "bitmap",
           "type": "Bitmap",
           "value": [
             {
               "name": "data",
               "type": "Stream",
               "value": "[]"
             }
           ]
         }
       ]
     },
     {
       "name": "pen",
       "type": "Pen",
       "value": [
         {
           "name": "width",
           "type": "float",
           "value": "1"
         },
         {
           "name": "dash",
           "type": "Dash",
           "value": "none"
         },
         {
           "name": "join",
           "type": "Join",
           "value": "miter"
         },
         {
           "name": "cap",
           "type": "LineCap",
           "value": "butt"
         }
       ]
     },
     {
       "name": "font",
       "type": "Font",
       "value": [
         {
           "name": "size",
           "type": "float",
           "value": "32"
         },
         {
           "name": "name",
           "type": "string",
           "value": "/usr/share/fonts/truetype/FreeMono.ttf"
         }
       ]
     },
     {
       "name": "justify",
       "type": "Justify",
       "value": "left"
     }
   ]
}

that's insane.


More information about the Digitalmars-d mailing list