stream interfaces - with ranges

[Christophe Travert]

I don't have time to read the whole discussion right now, but I've 
thought since our exchange here about buffered stream. I've imagined 
something close to, but quite different from you buffered stream, where 
the length of the buffer chunk can be adapted, and the buffer be poped 
by an arbitrary amount of bytes:

I reuse the name front, popFront and empty, but it may not be such a 
good idea.

struct BufferedStream(T)
{
  T[] buf;
  size_t cursor;
  size_t decoded;
  InputStream input;

  // returns a slice to the n next elements of the input stream.
  // this slice is valid until next call to front only.
  T[] front(size_t n)
  {
    if (n <= decoded - cursor) return buf[cursor..cursor+n];
    if (n <= buffer.length)
      {
       ... // move data to the front of the buffer and read new data to 
           // fill the buffer.
        return buf[0..n];
      }
    if (n > buf.length)
     {
       ... // resize buffer and read new data to fill the buffer
       return buf[0..n];
     }
  }
  // pop the next n elements from the buffer.
  void popFront(size_t n) { cursor += n; }
  void empty() { return input.eof && cursor == buf.length; }
}

This kind of buffered stream enable you read data by varying chunk size, 
but always read data by an amount that is convenient for the input 
stream. (and front could be made to return a buffer with the size that 
is most adequate for the stream when called with size_t.max as n).

More importantly, it allows to peak at an arbitrary amount of data, use 
it, and decide how many items you want to consume. For example, if 
allows to write stuff like "ReadAWord" without double buffering: you 
get enough characters from the buffer until you find a space, and then 
you consume only the characters that are the space.

"Steven Schveighoffer" , dans le message (digitalmars.D:167733), a
 écrit :
> OK, so I had a couple partially written replies on the 'deprecating
> std.stream etc' thread, then I had to go home.
> 
> But I thought about this a lot last night, and some of the things Andrei
> and others are saying is starting to make sense (I know!).  Now I've
> scrapped those replies and am thinking about redesigning my i/o package
> (most of the code can stay intact).
> 
> I'm a little undecided on some of the details, but here is what I think
> makes sense:
> 
> 1. We need a buffering input stream type.  This must have additional
> methods besides the range primitives, because doing one-at-a-time byte
> reads is not going to cut it.
> 2. I realized, buffering input stream of type T is actually an input range
> of type T[].  Observe:
> 
> struct /*or class*/ buffer(T)
> {
>       T[] buf;
>       InputStream input;
>       ...
>       @property T[] front() { return buf; }
>       void popFront() {input.read(buf);} // flush existing buffer, read  
> next.
>       @property bool empty() { return buf.length == 0;}
> }
> 
> Roughly speaking, not all the details are handled, but this makes a
> feasible input range that will perform quite nicely for things like
> std.algorithm.copy.  I haven't checked, but copy should be able to handle
> transferring a range of type T[] to an output range with element type T,
> if it's not able to, it should be made to work. 

Or with joiner(buffer);

> I know at least, an
> output stream with element type T supports putting T or T[].  What I think
> really makes sense is to support:
> 
> buffer!ubyte b;
> outputStream o;
> 
> o.put(b); // uses range primitives to put all the data to o, one element
> (i.e. ubyte[]) of b at a time

Of course, output stream should not have a consistent interface with 
input stream.

> 3. An ultimate goal of the i/o streaming package should be to be able to
> do this:
> 
> auto x = new XmlParser("<rootElement></rootElement>");
> 
> or at least
> 
> auto x = new XmlParser(buffered("<rootElement></rootElement>"));
> 
> So I think arrays need to be able to be treated as a buffering streams.  I
> tried really hard to think of some way to make this work with my existing
> system, but I don't think it will without unnecessary baggage, and losing
> interoperability with existing range functions.

A simple string stream can be built on top of a string, with no 
other member than the string itself, can't it ?
With my definition of buffered stream, at least, it can, and any array 
could support:
T[] front(size_t i) { return this[0..min(i, $)]; }
void popFront(size_t i) { this = this[i..$]; }

-- 
Christophe