[RFC] I/O and Buffer Range

Dmitry Olshansky dmitry.olsh at gmail.com
Fri Jan 17 02:01:35 PST 2014


17-Jan-2014 02:41, Steven Schveighoffer пишет:
> On Thu, 16 Jan 2014 17:28:31 -0500, Dmitry Olshansky
> <dmitry.olsh at gmail.com> wrote:
>
>>
>> The other way around :) 4 code units - 1 code point.
>
> I knew that was going to happen :)

Aye. BTW I haven't thought of writing into the buffer, but it works 
exactly the same. It could be even read/write -"discarded" data is 
written to underlying stream, freshly loaded is read from stream. Now in 
case of input stream only writes are nop, for output-only reads are nops.

With pinning it makes for cool multi-pass algorithms that actually 
output stuff into the file.

>>> This would be a key addition for ANY type in order to properly work with
>>> shared. BUT, I don't see how it works safely generically because you
>>> necessarily have to cast away shared in order to call the methods. You
>>> would have to limit this to only working on types it was intended for.
>>
>> The requirement may be that it's pure or should I say
>> "well-contained". In other words as long as it doesn't smuggle
>> references somewhere else it should be fine.
>> That is to say it's 100% fool-proof, nor do I think that essentially
>> simulating a synchronized class is a always a good thing to do...
>
> I think you meant *not* 100% :) But yeah, regardless of how generalized
> it is, this is likely the best path. I think this is the tack that
> std.stdio.File takes anyway, except it's using FILE *'s locking mechanism.
>

Aye.

>> Convenient to work with does ring good to me. I simply see no need to
>> reinvent std.algorithm on buffers especially the ones that just scan
>> left-to-right.
>> Example would be calculating a checksum of a stream (say data comes
>> from a pipe or socket i.e. buffered). It's a trivial application of
>> std.algorithm.reduce and there no need to reinvent that wheel IMHO.
>>
>
> I again think I am being misunderstood. Example might be appropriate:
>

Looking at the post by Walter I see I need to clarify things.
And if you browse the thread you'd see my understanding also changed 
with time - I started with no stinkin' forward ranges with buffered I/O 
only to later eat my words and make them happen.

First let's call buffer a thing that pack an array and few extras to 
support pinning, refiling of data from underlying stream and extending.
It exposes current "window" of underlying stream.

Now we have pins in that buffer that move along. A pin not only enforces 
that all data "to the left" stays accessible but also is a "view" of 
buffer. It's even conceptually a range with extra primitives outlined here:
http://blackwhale.github.io/datapicked/dpick.buffer.traits.html

I should stick to naming it BufferRange. The "real buffer" is internal 
construct and BufferRanges share ownership of it.

This is exactly what I ended up with in my second branch.
"real" buffer:
https://github.com/blackwhale/datapicked/blob/fwd-buffer-range/dpick/buffer/buffer.d#L417
vs buffer range over it:
https://github.com/blackwhale/datapicked/blob/fwd-buffer-range/dpick/buffer/buffer.d#L152

Naming issues are apparently even worse then Walter implies.

> struct Buffer {...} // implements BOTH forward range and Buffer primitives
>
> struct OtherBuffer {...} // only implements Buffer primitives.
>
> If isBuffer is modified to not require isForwardRange, then both Buffer
> and OtherBuffer will work as buffers, but only Buffer works as a range.
>
> But as you have it now, isBuffer!OtherBuffer is false. Is this necessary?
>

I think I should call it BufferRange from now on. And bring my docs in 
line. It may make sense to provide naked Buffer itself as a construct 
(it has simpler interface) and have generic BufferRange wrapper. I'm 
just not seeing user code using the former - too error prone and unwieldy.

> So we can implement buffers that are both ranges and buffers, and will
> work with std.algorithm without modification (and that's fine and
> expected by me), but do we need to *require* that? Are we
> over-specifying?

Random-Access range had to require .front/.back maybe it was 
over-specification too? Some stuff is easy to index but not "drop off an 
item at either end". But now it really doesn't matter - if there are 
such things, they are not random-access ranges.

> Is there a possibility that someone can invent a buffer
> that satisfies the primitives of say a line-by-line reader, but cannot
> possibly be a forward range?

I hardly can see that:
front --> lookahead(1)[0]
empty --> lookahead(1).length != 0
popFront --> seek(1) or enforce(seek(1))

save -> well there got to be a way to pin the data in the buffer?

And they surely could be better implemented inside of a specific buffer 
range.

-- 
Dmitry Olshansky


More information about the Digitalmars-d mailing list