Request for review - std.serialization (orange)

[Matt Soucy]

On 04/01/2013 01:13 PM, Jesse Phillips wrote:
> On Monday, 1 April 2013 at 08:53:51 UTC, Jacob Carlborg wrote:
>> On 2013-04-01 01:39, Jesse Phillips wrote:
>>
>>> I'm not well versed in PB or Orange so I'd need to play around more with
>>> both, but I'm pretty sure Orange would need changes made to be able to
>>> make the claim PB is supported. It should be possible to create a binary
>>> format based on PB.
>>
>> Isn't PB binary? Or it actually seems it can be both.
>
> Let me see if I can describe this.
>
> PB does encoding to binary by type. However it also has a schema in a
> .proto file. My first concern is that this file provides the ID to use
> for each field, while arbitrary the ID must be what is specified.
>
> The second one I'm concerned with is option to pack repeated fields. I'm
> not sure the specifics for this encoding, but I imagine some compression.
>
> This is why I think I'd have to implement my own Serializer to be able
> to support PB, but also believe we could have a binary format based on
> PB (which maybe it would be possible to create a schema of Orange
> generated data, but it would be hard to generate data for a specific
> schema).
 From what I got from the examples, Repeated fields are done roughly as 
following:
auto msg = fields.map!(a=>a.serialize())().reduce!(a,b=>a~b)();
return ((id<<3)|2) ~ msg.length.toVarint() ~ msg;
Where msg is a ubyte[].
-Matt Soucy