stream interfaces - with ranges

[Andrei Alexandrescu]

On 5/18/12 8:51 AM, kenji hara wrote:
> OK. If reading bytes from underlying device failed, your 'fronts' can
> return empty slice. I understood.
> But, It is still *not efficient*. The returned slice will specifies a
> buffer controlled by underlying device. If you want to gather bytes
> into one chunk, you must copy bytes from returned slice to your chunk.
> We should reduce copying memories as much as possible.

Yah, this goes back to the fact that ranges are by definition buffered; 
there's no way to escape that.

So as I said, we need to add unbuffered primitives (e.g. "Here's a 
buffer, fill it with data). They would work with both inputs that have 
no buffering at all, and with ranges.

> And, 'put' primitive in output range concept doesn't support non-blocikng write.
> 'put' should consume *all* of given data and write it  to underlying
> device, then it would block.

Right.

Zero-copy I/O is a possibility, we need to define primitives for 
destructively transferring buffers to and from streams/ranges.

> Therefore, whole of range concept doesn't cover non-blocking I/O.

Correct. Doesn't cover zero-copy I/O either.

It's interesting to think what primitives we should define, and what 
algorithms can take advantage of them beyond just copy().

>>> I'm designing experimental IO primitives:
>>> https://github.com/9rnsr/dio

Had only time to skim it, looks very promising.

> Range concept is good abstraction if underlying container controlls
> ownership. But, in I/O we want to *move* ownership of bytes. Range is
> not designed efficiently for the purpose, IMO.

Yes, yes, yes. Perfect thinking. (What ranges are good at though is for 
algorithms to mess with.)



Andrei