std.Stream.InputStream convenience functions

[Jonathan M Davis via Digitalmars-d]

On Saturday, 5 September 2015 at 19:59:03 UTC, bitwise wrote:
> Thanks for the explanation, but could you give an example of 
> how Stream would be rangified?
>
> In the containers I'm writing, I'm returning a range from the 
> findAll() function instead of an array:
> https://github.com/bitwise-github/d-containers/blob/master/collections/list.d#L901
>
> I suppose any method returning an array/string could work like 
> this...Is this what you mean?
>
> It doesn't seem like the entire Stream class should have to be 
> removed to make this change though.

I would have to study the problem to give a good answer. It's 
been too long since I looked at the issue for me to remember the 
finer details at this point. IIRC, a big part of the problem had 
something to do with the fact that you essentially want a range 
of bytes, but the underlying C APIs really give something more 
like a range of a range of bytes, because you read the data into 
a buffer rather than reading it a byte at a time (which gets even 
further complicated by the fact that you generally want to reuse 
that buffer and not allocate extra memory if you can avoid it). 
On some level at least, that's the problem that byLine and 
byChunk in std.stdio run into. I really don't remember the 
details of what needed to be done differently for solid stream 
support though.

But all of the stuff relating to getting objects out of a byte 
stream can almost certainly be done by building it on top of a 
range of bytes rather than having any of that talk to the disk or 
generally care where the data comes from, and I very much doubt 
that std.stream does anything like that. So, at minimum, what 
std.stream does would have to be shifted so that all of the 
transformations are done on top of ranges of ubyte rather than 
talking to the disk or network stack or whatever. But I'd have to 
go digging through the code to even know what std.stream does 
right now, and I don't have the time right now to figure out how 
it should work. It's been discussed before, and it was decided 
that what's there was not what we wanted, though how far off it 
is, I really don't know other than the fact that it's not 
range-based. Any stream solution should be consuming and 
producing ranges and should be agnostic to wherever the data 
comes from unless it has a _really_ good reason not to be.

Steven Schveighoffer has been working off and on on a replacement 
for std.stdio which supports streams and cleans up some of the 
uglier things with std.stdio.File, so we may get something out of 
that at some point, but he has yet to get far enough with it to 
actually present it for review and possible inclusion in Phobos. 
However, I haven't heard of any other work being done on streams, 
and it seems like everyone's simply making do without, for better 
or worse.

- Jonathan M Davis