std.stream replacement

[BLM768]

On Wednesday, 6 March 2013 at 16:36:38 UTC, Steven Schveighoffer 
wrote:
> On Tue, 05 Mar 2013 18:24:22 -0500, BLM768 <blm768 at gmail.com> 
> wrote:
>>
>> Ranges aren't necessarily higher- or lower-level than streams; 
>> they're completely orthogonal ways of looking at a data 
>> source. It's completely possible to build a stream interface 
>> on top of a range of characters, which is what I was 
>> suggesting. In that situation, the range is at a lower level 
>> of abstraction than the stream is.
>
> I think you misunderstand.  Ranges absolutely can be a source 
> for streams, especially if they are arrays.  The point is that 
> the range *interface* doesn't make a good stream interface.  So 
> we need to invent new methods to access streams.

Although I probably didn't communicate it very well, my idea was 
that since we already have functions like std.conv.parse that 
essentially provide parts of a stream interface on top of ranges, 
the most convenient way to implement a stream might be to build 
it on top of a range interface so no code duplication is needed.

>> Create a range operation like "r.takeArray(n)". You can 
>> optimize it to take a slice of the buffer when possible.
>
> This is not a good idea.  We want streams to be high 
> performance.  Accepting any range, such as a dchar range that 
> outputs one dchar at a time, is not going to be high 
> performance.

If the function is optimized, it can essentially bypass the range 
layer and operate directly on the buffer while using the same 
interface it would use if it were operating on the range. As I 
understand it, some of the operations in Phobos do that as well 
when given arrays.

>
> On top of that, in some cases, the result will be a slice, in 
> some cases it will be a copy.  Generic code will have to figure 
> out that difference if it wants to save the data for later, or 
> else risk double copying.
>

That could definitely be an issue. It should be possible to 
enforce slicing semantics somehow, but I'd have to think about it.

>>
>> Range operations like std.conv.parse implicitly progress their 
>> source ranges.
>
> That's not a range operation.  Range operations are empty, 
> popFront, front.  Anything built on top of ranges must use ONLY 
> these three operations, otherwise you are talking about 
> something else.

I guess that's not the right terminology for what I'm trying to 
express. I was thinking of "operations that act on ranges."


>> From what I see, at least in terms of the interface, a stream 
>> is basically just a generalization of a range that supports 
>> more than one type as input/output. There's no reason that 
>> such a system couldn't be built on top of a range, especially 
>> when the internal representation is of a single type: 
>> characters.
>
> streams shouldn't have to support the front/popFront mechanism.
>  empty may be the only commonality.  I think that is an awkward 
> fit for ranges.  Certainly it is possible to take a *specific* 
> range, such as an array, and add a stream-like interface to it.
>  But not ranges in general.

I hadn't considered the case of r.front; I was only thinking 
about r.popFront. Looks like they're a little more different than 
I was thinking, but they're still very similar under certain 
conditions.

Ultimately, we do need some type of a traditional stream 
interface; I was just thinking about using ranges behind the 
scenes and using existing pieces of the standard library for 
stream operations rather than putting all of the operations into 
a unified data type. I'm not sure if it could really be called an 
"ideal" design, but I do think that it could provide a good 
minimalist solution with performance that would be acceptable for 
at least many applications.