stream == range ? [Sliding window]

[Dmitry Olshansky via Digitalmars-d]

On 01-Jun-2015 02:54, Dragos Carp wrote:
>>>> Hardly. In fact, I've spent quite some time trying to figure this out.
>>>>
>> 2. As it stands it would require first copying into appender which is
>> slow. The opposite direction is usually a better match - assume
>> appender (or rather buffer) is filled to its capacity with data from
>> the file and keeping this invariant true.
>
> This would function differently if the data comes over the socket,
> usually you cannot fill the buffer.

Agreed, with sockets it's not going to work like that.

>>
>> Otherwise it looks quite similar to the sliding window interface and
>> that is great.
>>
>> data --> window
>> popFrontN --> skip
>> put ~~> extend (probably should be just called 'more')
>
> Sometimes (socket recv) you don't know how much extend.
>
That's still OK, if extend is "try to load at least this much". That is 
it attempts a read of max(capacity_left, bytes_to_extend) and adjusts 
the visible window.

[snip]
>>
>> Parser or any other component down the line shouldn't concern itself
>> with reading, just using some "more" primitive to auto-magically
>> extend the window by at least X bytes. That would shrink-extend
>> buffer  and load it behind the scenes.
>
> As I sad, LinearBuffer is meant to be used "behind the scenes".
>

Okay, then we are on the same page.

I think we need to define a set of primitives and semantics that define 
a buffer concept, it's seems that 3 primitives outlined is enough 
plus/minus some traits.

> I find Andrei's bulkRead, maybe combined with a matcher a la boost::asio
> read_until, the simplest high level interface.

For unbuffered streams - read_until is probably too much, a bulkRead 
would be the minimum.

For buffered streams - should be easily defined as stand-alone:

bool readUntil(alias matcher, BufStream)(BufStream buf){
	for(;;){
		auto pos = matcher(buf.window);
		if(pos < 0)
			buf.skip(buf.window.length);
			if(!buf.more()) return false;
		}
		else{
			buf.skip(pos);
			return true;
		}
	}
}

-- 
Dmitry Olshansky