Request for review - std.serialization (orange)

[Matt Soucy]

On 04/02/2013 12:38 AM, Kagamin wrote:
> On Monday, 1 April 2013 at 21:11:57 UTC, Matt Soucy wrote:
>> And therefore, it supports arrays just fine (as repeated fields). Yes.
>> That last sentence was poorly-worded, and should have said "you'd just
>> end up with the un'packed' data with an extra header."
>
> It says repeated messages should be merged which results in one message,
> not an array of messages. So from several repeated messages you get one
> as if they formed contiguous soup of fields which got parsed as one
> message: e.g. scalar fields of the resulting message take their last
> seen values.
>

They're merged if the field is optional and gets multiple inputs with 
that id. If a field is marked as repeated, then each block of data 
denoted with that field is treated as a new item in the array.

>> Unfortunately, I'm not particularly knowledgeable about networking,
>> but that's not quite what I meant. I meant that the use case itself
>> would result in sending individual Result messages one at a time,
>> since packing (even if it were valid) wouldn't be useful and would
>> require getting all of the Results at once. You would just leave off
>> the "packed" attribute.
>
> As you said, there's no way to tell where one message ends and next
> begins. If you send them one or two at a time, they end up as a
> contiguous stream of bytes. If one is to delimit messages, he should
> define a container format as an extension on top of PB with additional
> semantics for representation of arrays, which results in another
> protocol. And even if you define such protocol, there's still no way to
> have array fields in PB messages (arrays of non-trivial types).
>
> For example if you want to update students and departments with one
> method, the obvious choice is to pass it a dictionary of key-value pairs
> of new values for the object's attributes. How to do that?

I said that that only applies to the incorrect idea of packing complex 
messages. With regular repeated messages (nonpacked), each new repeated 
message is added to the array when deserialized.
While yes, protocol buffers do not create a way to denote 
uppermost-level messages, that isn't really relevant to the situation 
that you're trying to claim. If messages are supposed to be treated 
separately, there are numerous ways to handle that which CAN be done 
inside of protocol buffers.
In this example, one way could be to define messages like so:

message Updates {
	message StudentUpdate {
		required string studentName = 1;
		required uint32 departmentNumber = 2;
	}
	repeated StudentUpdate updates = 1;
}

The you would iterate over Updates.updates, which you'd be adding to 
upon deserialization of more of the messages.