deprecating std.stream, std.cstream, std.socketstream

[H. S. Teoh]

On Wed, May 16, 2012 at 04:52:09PM -0400, Steven Schveighoffer wrote:
> On Wed, 16 May 2012 16:30:43 -0400, H. S. Teoh
> <hsteoh at quickfur.ath.cx> wrote:
> 
> >On Wed, May 16, 2012 at 04:15:22PM -0400, Steven Schveighoffer wrote:
> >>On Wed, 16 May 2012 15:38:02 -0400, H. S. Teoh
> >><hsteoh at quickfur.ath.cx> wrote:
> >[...]
> >>>One direction that _could_ be helpful, perhaps, is to extend the
> >>>concept of range to include, let's tentatively call it, a
> >>>ChunkedRange.  Basically a ChunkedRange implements the usual
> >>>InputRange operations (empty, front, popfront) but adds the
> >>>following new primitives:
> >>>
> >>>- bool hasAtLeast(R)(R range, int n) - true if underlying range has
> >>>  at least n elements left;
> >>>
> >>>- E[] frontN(R)(R range, int n) - returns a slice containing the
> >>>  front n elements from the range: this will buffer the next n
> >>>  elements from the range if they aren't already; repeated calls
> >>>  will just return the buffer;
> >>>
> >>>- void popN(R)(R range, int n) - discards the first n elements from
> >>>  the buffer, thus causing the next call to frontN() to fetch more
> >>>  data if necessary.
> >>>
> >>
> >>On such ranges, what would popFront and front do?  I'm assuming
> >>since frontN and popN are referring to how many elements, and since
> >>the most logical definition for elements is bytes, that front gets
> >>the next byte, and popFront discards the next byte.  This seems
> >>useless to me.
> >
> >How so? It's still useful for implementing readByte, for example.
> 
> readByte is covered by frontN(1).  Why the need for front()?
> 
> Let me answer that question for you -- so it can be treated as a
> normal range.  But nobody will want to do that.
> 
> i.e. copy to appender will read one byte at a time into the array.

If this new type of range is recognized by std.range, then the relevant
algorithms can be made to recognize the existence of frontN and make
good use of it, instead of iterating front N times. Then front() can
still be used by stuff that really only wants a single byte at a time.


[...]
> >The idea is that by asking for N elements at a time instead of
> >calling front/popFront N times, the underlying implementation can
> >optimize the request by creating a buffer of size N and have the OS
> >read exactly N bytes directly into that buffer.
> >
> >	// Reads 1,000,000 bytes into newly allocated buffer and returns
> >	// buffer.
> >	auto buf = stream.frontN(1_000_000);
> 
> OK, so stream is providing data via return value and allocation.
> 
> >	// Since 1,000,000 bytes is already read into the buffer, this
> >	// simply returns a slice of the same buffer:
> >	auto buf2 = stream.frontN(1_000_000);
> 
> Is buf2 mutable?  If so, this is no good, buf could have mutated
> this data.  But this can be fixed by making the return value of
> frontN be const(ubyte)[].
> 
> >	assert(buf is buf2);
> >
> >	// This consumes the buffer:
> >	stream.popN(1_000_000);
> 
> What does "consume" mean, discard?  Obviously not "reuse", due to
> line below...

Yes, discard. That's what popFront does right now for a single element.


> >	// This will read another 1,000,000 bytes into a new buffer
> >	auto buf3 = stream.frontN(1_000_000);
> 
> OK, you definitely lost me here, this will not fly.  The whole point
> of buffering is to avoid having to reallocate on every read.  If you
> have to allocate every read, "buffering" is going to have a negative
> impact on performance!
[...]

I thought the whole point of buffering is to avoid excessive roundtrips
to disk I/O.

Though you do have a point that allocating on every read is a bad idea.


T

-- 
Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? -- Michael Beibl