Request for review - std.serialization (orange)

[Matt Soucy]

On 04/02/2013 10:52 AM, Jacob Carlborg wrote:
> On 2013-04-02 15:38, Matt Soucy wrote:
>
>> Unfortunately, only partially correct. Optional isn't an "option", it's
>> a way of saying that a field may be specified 0 or 1 times. If two
>> messages with the same ID are read and the ID is considered optional in
>> the schema, then they are merged.
>
> With "option", I mean you don't have to use it in the schema. But the
> (de)serializer of course need to support this to be fully compliant with
> the spec.
>

OK, I see what you mean. PB uses the term "option" for language 
constructs, hence my confusion.

>> Packed IS an "option", which can only be done to primitives. It changes
>> serialization from:
>>  > return raw.map!(a=>(MsgType!BufferType | (id << 3)).toVarint() ~
>> a.writeProto!BufferType())().reduce!((a,b)=>a~b)();
>> to
>>  > auto msg = raw.map!(writeProto!BufferType)().reduce!((a,b)=>a~b)();
>>  > return (2 | (id << 3)).toVarint() ~ msg.length.toVarint() ~ msg;
>>
>> (Actual snippets from my partially-complete protocol buffer library)
>>
>> If you had a struct that matches that schema (PB messages have value
>> semantics) then yes, in theory you could do something to serialize the
>> struct based on the schema, but you'd have to maintain both separately.
>
> Just compile the schema to a struct with the necessary fields. Perhaps
> not how it's usually done.
>

Again, my misunderstanding. I assumed you were talking about taking a 
pre-existing struct, not one generated from the .proto

>> PB is NOT dependent on the order of the fields during serialization,
>> they can be sent/received in any order. You could use the schema like
>> you mentioned above to tie member names to ids, though.
>
> So if you have a schema like this:
>
> message Person {
>    required string name = 1;
>    required int32 id = 2;
>    optional string email = 3;
> }
>
> 1, 2 and 3 will be ids of the fields, and also the order in which they
> are (de)serialized?
>
> Then you could have the archive read the schema, map names to ids and
> archive the ids instead of the names.
>

You could easily receive 3,1,2 or 2,1,3 or any other such combination, 
and it would still be valid. That doesn't stop you from doing what you 
suggest, however, as long as you can lookup id[name] and name[id].

>> PB uses value semantics, so multiple references to the same thing isn't
>> really an issue that is covered.
>
> I see, that kind of sucks, in my opinion.
>

Eh. I personally think that it makes sense, and don't have much of a 
problem with it.