Streaming transport interfaces: input

Andrei Alexandrescu SeeWebsiteForEmail at erdani.org
Thu Oct 14 21:37:55 PDT 2010


On 10/14/10 21:58 CDT, Steven Schveighoffer wrote:
> On Thu, 14 Oct 2010 16:47:13 -0400, Steven Schveighoffer
> <schveiguy at yahoo.com> wrote:
>
>> On Thu, 14 Oct 2010 14:43:56 -0400, Andrei Alexandrescu
>
>>> How? Denis' implementation has two copies in the mix. (I'm not
>>> counting .dup etc.) Anyhow, let's do this - write down your
>>> interfaces so I can comment on them. We talk "oh that's a buffering
>>> interface" and "that requires buffering" and "that's an extra copy"
>>> and so on but we have little concrete contenders. I put my cards on
>>> the table, you put yours.
>>
>> I'll see if I can put something together.
>
> Here's a rough outline:

Thanks!

> enum Anchor
> {
> Begin,
> Current,
> End
> }
>
> interface Seek
> {
> ulong seek(long delta, Anchor whence);
> final ulong tell() { return seek(0, Anchor.Current); }
> bool seekable(); // define as false if seeking is not supported, true if it
> // is supported (this doesn't necessarily mean a seek will
> // succeed).
> }

So far so good.

> interface InputTransport : Seek
> {
> size_t read(ubyte[] data); // returns 0 on EOF.
> }

No way to check for end of stream except by reading some of it?

> // defined to implement either a D buffered object or wrap a FILE *.
> //
> interface BufferedInputTransport : Seek
> {
> size_t read(ubyte[] data); // returns 0 on EOF.

Since this method has the same sig, why doesn't BufferedInputTransport 
inherit InputTransport?

> // read data into the buffer until the delegate returns other than ~0
> //
> // The delegate is passed the entire buffer so far, with the start of the
> // new data just read. It returns other than ~0 when it determines the end
> // of the data in question.
> //
> ubyte[] readUntil(uint delegate(ubyte[] data, uint start) process);

How does the delegate say "you know what, I'm fine with the first 1000 
bytes of the data; please take the rest of 1048 back"? Is that the 
result of the delegate? The process feels a bit odd.

> // same as readUntil except append to the given arr, Any excess
> // data will be pushed into the internal buffer.
> //
> size_t appendUntil(uint delegate(ubyte[] data, uint start) process, ref
> ubyte[] arr)

So indeed the delegate seems to return the length it wants to keep? And 
the rest would be copied back into the stream's internal buffers? I'm 
not sure I understand this API.

> // various buffer functions.
> @property size_t bufsize();
> @property size_t readable();
> // etc.

Can one set bufsize?

> }
>
> The way I see it working is, there are two implementations for
> BufferedInputTransport: FILEInputTransport and DBufferInputTransport.
> There are numerous implementations of InputTransport, each of which can
> be passed to the DBufferInputTransport, which uses its own buffer
> implementation. For example, a network socket, file, inter-thread
> stream, an array, etc.
>
> This way, you can play nice with C's stdio when necessary (i.e. for
> stdin/stdout/stderr) and avoid the FILE limitations and performance
> issues otherwise.

I'm a bit unclear on the delegate stuff, but it's promising because it 
could be quite flexible. But I wouldn't want to aggravate the users with 
an API that's difficult to use. Could you please give a few examples 
using delegates that implement common patterns - e.g. readline and 
readDelim?


Andrei


More information about the Digitalmars-d mailing list