buffered input

[spir]

On 02/06/2011 01:28 AM, Don wrote:
> spir wrote:
>> On 02/05/2011 10:44 PM, Nick Sabalausky wrote:
>>> "Andrei Alexandrescu"<SeeWebsiteForEmail at erdani.org> wrote in message
>>> news:iijq99$1a5o$1 at digitalmars.com...
>>>> On 2/5/11 6:46 AM, spir wrote:
>>>>> On 02/05/2011 10:36 AM, Nick Sabalausky wrote:
>>>>>> On a separate note, I think a good way of testing the design (and end up
>>>>>> getting something useful anyway) would be to try to use it to create a
>>>>>> range
>>>>>> that's automatically-buffered in a more traditional way. Ie, Given any
>>>>>> input
>>>>>> range 'myRange', "buffered(myRange, 2048)" (or something like that)
>>>>>> would
>>>>>> wrap it in a new input range that automatically buffers the next 2048
>>>>>> elements (asynchronously?) whenever its internal buffer is exhausted. Or
>>>>>> something like that. It's late and I'm tired and I can't think anymore
>>>>>> ;)
>>>>>
>>>>> That's exactly what I'm expecting.
>>>>> Funnily enough, I was about to start a thread on the topic after reading
>>>>> related posts. My point was:
>>>>> "I'm not a specialist in efficiency (rather the opposite), I just know
>>>>> there is --theoretically-- relevant performance loss to expect from
>>>>> unbuffered input in various cases. Could we define a generic
>>>>> input-buffering primitive allowing people to benefit from others'
>>>>> competence? Just like Appender."
>>>>>
>>>>> Denis
>>>>
>>>> The buffered range interface as I defined it supports infinite lookahead.
>>>> The interface mentioned by Nick has lookahead between 1 and 2048. So I
>>>> don't think my interface is appropriate for that.
>>>>
>>>> Infinite lookahead is a wonderful thing. Consider reading lines from a
>>>> file. Essentially what you need to do is to keep on reading blocks of data
>>>> until you see \n (possibly followed by some more stuff). Then you offer
>>>> the client the line up to the \n. When the client wants a new line, you
>>>> combine the leftover data you already have with new stuff you read. On
>>>> occasion you need to move over leftovers, but if your internal buffers are
>>>> large enough that is negligible (I actually tested this recently).
>>>>
>>>> Another example: consider dealing with line continuations in reading CSV
>>>> files. Under certain conditions, you need to read one more line and stitch
>>>> it with the existing one. This is easy with infinite lookahead, but quite
>>>> elaborate with lookahead 1.
>>>>
>>>
>>> I think I can see how it might be worthwhile to discourage the traditional
>>> buffer interface I described in favor of the above. It wouldn't be as
>>> trivial to use as what people are used to, but I can see that it could avoid
>>> a lot of unnessisary copying, especially with other people's suggestion of
>>> allowing the user to provide their own buffer to be filled (and it seems
>>> easy enough to learn).
>>>
>>> But what about when you want a circular buffer? Ie, When you know a certain
>>> maximum lookahead is fine and you want to minimize memory usage and
>>> buffer-appends. Circular buffers don't do infinite lookahead so the
>>> interface maybe doesn't work as well. Plus you probably wouldn't want to
>>> provide an interface for slicing into the buffer, since the slice could
>>> straddle the wrap-around point which would require a new allocation (ie
>>> "return buffer[indexOfFront+sliceStart..$] ~
>>> buffer[0..sliceLength-($-(frontIndex+sliceStart))]"). I guess maybe that
>>> would just call for another type of range.
>>
>> Becomes too complicated, doesnt it?
>>
>> Denis
>
> Circular buffers don't seem like an 'optional' use case to me. Most real I/O
> works that way.
> I think if the abstraction can't handle it, the abstraction is a failure.

Sorry, I meant the way we start to draw the picture; not circular buffers, can 
see the point about them. Think Heywood's "view window" is a helpful image and 
a good modelling starting point. (maybe it's only me)

Denis
-- 
_________________
vita es estrany
spir.wikidot.com