Another new io library

[Chad Joan via Digitalmars-d]

On Friday, 19 February 2016 at 01:29:15 UTC, Steven Schveighoffer 
wrote:
> On 2/18/16 6:52 PM, Chad Joan wrote:
>> ...
>>
>> This is why I think it will be much more important to have at 
>> least
>> these two interfaces take front-and-center:
>> A.  The presence of a .popAs!(...) operation (mentioned by 
>> Wyatt in this
>> thread, IIRC) for simple deserialization, and maybe for other
>> miscellaneous things like structured user interaction.
>
> To me, this is a higher-level function. popAs cannot assume to 
> know how to read what it is reading. If you mean something like 
> reading an entire struct in binary form, that's not difficult 
> to do.
>

I think I understand what you mean.  We are entering the problem 
domain of serializing and deserializing arbitrary types.

I think what I'd expect is to have the basic language types 
(ubyte, int, char, string, etc) all covered, and to provide some 
way (or ways) to integrate with serialization code provided by 
other types.  So you can do ".popAs!int" out of the box, but 
".popAs!MyType" will require MyType to provide a .deserialize 
member function.  Understandably, this may require some thought 
(ex: what if MyType is already under constraints from some other 
API that expects serialization? what does this look like if there 
are multiple serialization frameworks? etc etc).  I don't have 
the answer right now and I don't expect it to be solved quickly ;)

>> B.  The ability to attach parsers to streams easily.  This 
>> might be as
>> easy as coercing the input stream into the basic encoding that 
>> the
>> parser expects (ex: char/wchar/dchar Ranges for compilers, or 
>> maybe
>> ubyte Ranges for our PostgreSQL client's network layer), 
>> though it might
>> need (A) to help a bit first if the encoding isn't known in 
>> advance
>> (text files can be represented in sooo many ways!  isn't it 
>> fabulous!).
>
> This is the fundamental goal for my library -- enabling parsers 
> to read data from a "stream" efficiently no matter how that 
> data is sourced. I know your time is limited, but I would 
> invite you to take a look at the convert program example that I 
> created in my library. In it, I handle converting any UTF 
> format to any other UTF format.
>
> https://github.com/schveiguy/iopipe/blob/master/examples/convert/convert.d
>

Awesome!

>>
>> I understand that most unsuspecting programmers will arrive at 
>> a stream
>> library expecting to immediately see an InputRange interface.  
>> This
>> /probably/ is not what they really want at the end of the day.
>>  So, I
>> think it will be very important for any such library to 
>> concisely and
>> convincingly explain the design methodology and rationale 
>> early and
>> aggressively.  Neglect to do this, and the library and it's
>> documentation will become a frustration and a violation of 
>> expectations
>> (an "astonishment"). Do it right, and the library's 
>> documentation will
>> become a teaching tool that leaves visitors feeling 
>> enlightened and
>> empowered.
>
> Good points! I will definitely spend some time explaining this.
>

Best of luck :)

>> Of course, I have to wonder if someone else has contrasting 
>> experiences
>> with stream use-cases.  Maybe they really would be frustrated 
>> with a
>> range-agnostic design.  I don't want to alienate this 
>> hypothetical
>> individual either, so if this is you, then please share your 
>> experiences.
>>
>> I hope this helps and is worth making a bunch of you read a 
>> wall of text ;)
>
> Thanks for taking the time.
>
> -Steve

Thank you for making progress on this problem!

- Chad