Range interface for std.serialization

[Dmitry Olshansky]

26-Aug-2013 15:23, Jacob Carlborg пишет:
> On 2013-08-26 11:23, Dmitry Olshansky wrote:
>
>> Array or any container should do for a range.
>
> But then it won't be lazy, or perhaps that's not a problem, since the
> whole deserializing should be lazy.

It's lazy but you have to put stuff somewhere.
Also 2nd API artifact unpackRange allows you to just look through the 
data (output range including lambdas can do anything). This can easily 
sift through say swaths of data picking only "lucky numbers":

deserilaizer.unpackRange!(int)((x){
	if(isLucky(x))
		writeln(x);
});

>> I'm not 100% sure what kind of interface to use, but Serializer and
>> Deserializer should not be "shipped in one package" as in one class.
>> The two a mirror each other but in essence are always used separately.
>> Ditto about archiver/unarchiver they simply provide different
>> functionality and it makes no sense to reuse the same object in 2 ways.
>>
>> Hence alternative 1 (unwrapping that snippet backwards):
>>
>> //BTW why go new & classes here(?)
>
> The reason to have classes is that I need reference types. I need to
> pass the serializer to "toData" and "fromData" methods that can be
> implemented on the objects being (de)serialized. I guess they could take
> the argument by ref. Is it possible to force that?

Would be interesting to do that. One way is to pass rvalue to said 
function and if it accepts that then it's not by ref.

Along the lines of
__traits(compiles, (){
	T val;
	//or better can use dummy function
	//that returns Serializer by value
	val.toData(Serializer.init);
});

At least that works with templated stuff too.

>
>> auto unarchiver = new XmlUnarchiver(someCharRange);
>> auto deserialzier = new Deserializer(unarchiver);
>> auto obj = deserializer.unpack!Object;
>>
>> //for sequence/array in underlying format it would use any container
>> List!int list = deserializer.unpack!(List!int);
>> int[] arr = deserializer.unpack!(int[]);
>>
>> IMO looks quite nice. The problem of how exactly should a container be
>> filled is open though.
>>
>> So another alternative being more generic (the above could be consider
>> convenience over this one):
>>
>> Vector!int ints;
>> deserilaizer.unpackRange!(int)(x => ints.pushBack(x));
>>
>> Basically unpack next sequence of data (as serialized) by feeding it to
>> an output range using element type as param. And a simple lambda
>> qualifies as an output range.
>
> Here we have yet another suggestion for an API.

It's not just yet another. It isn't about particular shade of color. I 
can explain the ifs and whys of any design decision here if there is a 
doubt. I don't care for names but I see the precise semantics and there 
is little left to define.

For instance I see good reasons why serializer _has_ to be OutputRange 
and not InputRange. Why archiver _has_ to take output range or be one 
and so on. Ditto on why there has to be separation of (un)archiver and 
(de)serializer.

> The whole reason for
> this thread is that people weren't happy with the current interface,
> i.e. not range based. Now we got probably just as many suggestions as
> people who have answered to this thread. I still don't know hot the API
> should look like.

It's not a question of putting here some ranges. Or replacing all arrays 
one comes across to ranges(as much as a lot of folks would unfortunately 
assume).

Rather it's how it could operate with them at all without sacrificing 
the functionality, performance and ease of use. And there is not much in 
this tight design space that actually works.

Pardon me if my tone is a bit sharp. I like any other want the best 
design we can get. Now that the great deal of work is done it would be a 
shame to present it in a bad package.

>> Also take a look at the new digest API. I have an understanding that
>> serialization would do well to take the same general strategy - concrete
>> archivers as structs + polymorphic interface and wrappers on top.
>
> I could have a look at that.