[RFC] I/O and Buffer Range

[Joseph Cassman]

On Tuesday, 31 December 2013 at 09:04:58 UTC, Dmitry Olshansky 
wrote:
> 31-Dec-2013 05:53, Joseph Cassman пишет:
>> On Sunday, 29 December 2013 at 22:02:57 UTC, Dmitry Olshansky 
>> wrote:
>
> I'm thinking there might be a way to bridge the new range type 
> with ForwardRange but not directly as defined at the moment.
>
> A possibility I consider is to separate a Buffer object (not a 
> range), and let it be shared among views - light-weight 
> buffer-ranges. Then if we imagine that these light-weight 
> buffer-ranges are working as marks (i.e. they pin down the 
> buffer) in the current proposal then they could be forward 
> ranges.
>
> I need to think on this, as the ability to integrate well with 
> forwardish algorithms would be a great improvement.

I think I now understand a bit better what you were thinking when 
you first posted

> input-source <--> buffer range <--> parser/consumer
>
> Meaning that if we can mix and match parsers with buffer 
> ranges, and buffer ranges with input sources we had grown 
> something powerful indeed.

Being able to wrap an already-in-use range object with the buffer 
interface as you do in the sample code 
(https://github.com/blackwhale/datapicked/blob/master/dgrep.d) is 
good for composability. Also allows for existing functionality in 
std.algorithm to be reused as-is.

I think the new range type could also be added directly to some 
new, or perhaps retrofitted into existing, code to add the new 
functionality without sacrificing performance. In that way the 
internal payload already used to get the data (say by the input 
range) could be reused without having to allocate new memory to 
support the buffer API.

As one idea of using a buffer range from the start, a function 
template by(T) (where T is ubyte, char, wchar, or dchar) could be 
added to std.stdio. It would return a buffer range object 
providing more functionality than byChunk or byLine while adding 
access to the entire stream of data in a file in a contiguous and 
yet efficient manner. Seems to help with the issues faced in 
processing file data mentioned in previous comments in this 
thread.

Joseph