deprecating std.stream, std.cstream, std.socketstream

Steven Schveighoffer schveiguy at yahoo.com
Wed May 16 13:52:09 PDT 2012


On Wed, 16 May 2012 16:30:43 -0400, H. S. Teoh <hsteoh at quickfur.ath.cx>  
wrote:

> On Wed, May 16, 2012 at 04:15:22PM -0400, Steven Schveighoffer wrote:
>> On Wed, 16 May 2012 15:38:02 -0400, H. S. Teoh
>> <hsteoh at quickfur.ath.cx> wrote:
> [...]
>> >One direction that _could_ be helpful, perhaps, is to extend the  
>> concept
>> >of range to include, let's tentatively call it, a ChunkedRange.
>> >Basically a ChunkedRange implements the usual InputRange operations
>> >(empty, front, popfront) but adds the following new primitives:
>> >
>> >- bool hasAtLeast(R)(R range, int n) - true if underlying range has at
>> >  least n elements left;
>> >
>> >- E[] frontN(R)(R range, int n) - returns a slice containing the front  
>> n
>> >  elements from the range: this will buffer the next n elements from  
>> the
>> >  range if they aren't already; repeated calls will just return the
>> >  buffer;
>> >
>> >- void popN(R)(R range, int n) - discards the first n elements from the
>> >  buffer, thus causing the next call to frontN() to fetch more data if
>> >  necessary.
>> >
>>
>> On such ranges, what would popFront and front do?  I'm assuming since
>> frontN and popN are referring to how many elements, and since the most
>> logical definition for elements is bytes, that front gets the next
>> byte, and popFront discards the next byte.  This seems useless to me.
>
> How so? It's still useful for implementing readByte, for example.

readByte is covered by frontN(1).  Why the need for front()?

Let me answer that question for you -- so it can be treated as a normal  
range.  But nobody will want to do that.

i.e. copy to appender will read one byte at a time into the array.

>> I still don't get the need to "add" this to ranges.  The streaming API
>> works fine on its own.
>>
>> But there is an omission with your proposed API regardless --
>> reading data is a mutating event.  It destructively mutates the
>> underlying data stream so that you cannot get the data again.  This
>> means you must double-buffer data in order to support frontN and
>> popN that are not necessary with a simple read API.
>>
>> For example:
>>
>> auto buf = new ubyte[1000000];
>> stream.read(buf);
>>
>> does not need to first buffer the data inside the stream and then
>> copy it to buf, it can read it from the OS *directly* into buf.
> [...]
>
> The idea is that by asking for N elements at a time instead of calling
> front/popFront N times, the underlying implementation can optimize the
> request by creating a buffer of size N and have the OS read exactly N
> bytes directly into that buffer.
>
> 	// Reads 1,000,000 bytes into newly allocated buffer and returns
> 	// buffer.
> 	auto buf = stream.frontN(1_000_000);

OK, so stream is providing data via return value and allocation.

> 	// Since 1,000,000 bytes is already read into the buffer, this
> 	// simply returns a slice of the same buffer:
> 	auto buf2 = stream.frontN(1_000_000);

Is buf2 mutable?  If so, this is no good, buf could have mutated this  
data.  But this can be fixed by making the return value of frontN be  
const(ubyte)[].

> 	assert(buf is buf2);
>
> 	// This consumes the buffer:
> 	stream.popN(1_000_000);

What does "consume" mean, discard?  Obviously not "reuse", due to line  
below...

> 	// This will read another 1,000,000 bytes into a new buffer
> 	auto buf3 = stream.frontN(1_000_000);

OK, you definitely lost me here, this will not fly.  The whole point of  
buffering is to avoid having to reallocate on every read.  If you have to  
allocate every read, "buffering" is going to have a negative impact on  
performance!

-Steve


More information about the Digitalmars-d mailing list