buffered input

Andrei Alexandrescu SeeWebsiteForEmail at erdani.org
Sat Feb 5 07:25:19 PST 2011


On 2/5/11 6:46 AM, spir wrote:
> On 02/05/2011 10:36 AM, Nick Sabalausky wrote:
>> On a separate note, I think a good way of testing the design (and end up
>> getting something useful anyway) would be to try to use it to create a
>> range
>> that's automatically-buffered in a more traditional way. Ie, Given any
>> input
>> range 'myRange', "buffered(myRange, 2048)" (or something like that) would
>> wrap it in a new input range that automatically buffers the next 2048
>> elements (asynchronously?) whenever its internal buffer is exhausted. Or
>> something like that. It's late and I'm tired and I can't think anymore ;)
>
> That's exactly what I'm expecting.
> Funnily enough, I was about to start a thread on the topic after reading
> related posts. My point was:
> "I'm not a specialist in efficiency (rather the opposite), I just know
> there is --theoretically-- relevant performance loss to expect from
> unbuffered input in various cases. Could we define a generic
> input-buffering primitive allowing people to benefit from others'
> competence? Just like Appender."
>
> Denis

The buffered range interface as I defined it supports infinite 
lookahead. The interface mentioned by Nick has lookahead between 1 and 
2048. So I don't think my interface is appropriate for that.

Infinite lookahead is a wonderful thing. Consider reading lines from a 
file. Essentially what you need to do is to keep on reading blocks of 
data until you see \n (possibly followed by some more stuff). Then you 
offer the client the line up to the \n. When the client wants a new 
line, you combine the leftover data you already have with new stuff you 
read. On occasion you need to move over leftovers, but if your internal 
buffers are large enough that is negligible (I actually tested this 
recently).

Another example: consider dealing with line continuations in reading CSV 
files. Under certain conditions, you need to read one more line and 
stitch it with the existing one. This is easy with infinite lookahead, 
but quite elaborate with lookahead 1.


Andrei


More information about the Digitalmars-d mailing list