std.stream replacement

[Steven Schveighoffer]

On Tue, 05 Mar 2013 18:24:22 -0500, BLM768 <blm768 at gmail.com> wrote:

> On Tuesday, 5 March 2013 at 16:12:24 UTC, Steven Schveighoffer wrote:
>> On Tue, 05 Mar 2013 03:22:00 -0500, Jonathan M Davis  
>> <jmdavisProg at gmx.com> wrote:
>>
>>>
>>> In general, a stream _is_ a range, making a lot of "stream" stuff  
>>> basically
>>> irrelevant. What's needed then is a solid, efficient range interface  
>>> on top of
>>> I/O (which we're lacking at the moment).
>>
>> This is not correct.  A stream is a good low-level representation of  
>> i/o.  A range is a good high-level abstraction of that i/o.
>
> Ranges aren't necessarily higher- or lower-level than streams; they're  
> completely orthogonal ways of looking at a data source. It's completely  
> possible to build a stream interface on top of a range of characters,  
> which is what I was suggesting. In that situation, the range is at a  
> lower level of abstraction than the stream is.

I think you misunderstand.  Ranges absolutely can be a source for streams,  
especially if they are arrays.  The point is that the range *interface*  
doesn't make a good stream interface.  So we need to invent new methods to  
access streams.

>> Ranges make terrible streams for two reasons:
>>
>> 1. r.front does not have room for 'read n bytes'.  Making it do that is  
>> awkward (e.g. r.nextRead = 20; r.front; // read 20 bytes)
>
> Create a range operation like "r.takeArray(n)". You can optimize it to  
> take a slice of the buffer when possible.

This is not a good idea.  We want streams to be high performance.   
Accepting any range, such as a dchar range that outputs one dchar at a  
time, is not going to be high performance.

On top of that, in some cases, the result will be a slice, in some cases  
it will be a copy.  Generic code will have to figure out that difference  
if it wants to save the data for later, or else risk double copying.

>> 2. ranges have separate operations for getting data and progressing  
>> data.  Streams by their very nature combine the two in one operation  
>> (i.e. read)
>
> Range operations like std.conv.parse implicitly progress their source  
> ranges.

That's not a range operation.  Range operations are empty, popFront,  
front.  Anything built on top of ranges must use ONLY these three  
operations, otherwise you are talking about something else.

It is possible to use random-access ranges for a valid stream source.  But  
that is not a valid stream interface, streams aren't random-access ranges.

> Besides, streams don't necessarily progress the data; C++ iostreams have  
> peek(), after all.

That is because the data is buffered.  At a low-level, we have to deal  
with the OS, which may not support peeking.

>  From what I see, at least in terms of the interface, a stream is  
> basically just a generalization of a range that supports more than one  
> type as input/output. There's no reason that such a system couldn't be  
> built on top of a range, especially when the internal representation is  
> of a single type: characters.

streams shouldn't have to support the front/popFront mechanism.  empty may  
be the only commonality.  I think that is an awkward fit for ranges.   
Certainly it is possible to take a *specific* range, such as an array, and  
add a stream-like interface to it.  But not ranges in general.

-Steve