Another new io library

[Rikki Cattermole via Digitalmars-d]

On 17/02/16 7:45 PM, Steven Schveighoffer wrote:
> It's no secret that I've been looking to create an updated io library
> for phobos. In fact, I've been working on one on and off since 2011 (ouch).
>
> After about 5 iterations of API and design, and testing out ideas, I
> think I have come up with something pretty interesting. It started out
> as a plan to replace std.stdio (and that did not go over well:
> https://forum.dlang.org/post/j3u0l4$1atr$1@digitalmars.com), in addition
> to trying to find a better way to deal with i/o. However, I've scaled
> back my plan of world domination to just try for the latter, and save
> tackling the replacement of Phobos's i/o guts for a later battle, if at
> all. It's much easier to reason about something new than to muddle the
> discussion with how it will break code. It's also much easier to build
> something that doesn't have to be a drop-in replacement of something so
> insanely complex.
>
> I also have been inspired over the last few years by various great
> presentations and libraries, two being Dmitry's proof-of-concept library
> to have buffers that automatically move/fill when more data is needed,
> and Andrei's std.allocator library. They have changed drastically the
> way I have approached this challenge.
>
> Therefore, I now have a new dub-based repository available for playing
> with: https://github.com/schveiguy/iopipe. First, the candy:
>
> - This is a piping library. It allows one to hook buffered i/o through
> various processors/transformers much like unix pipes or range
> functions/algorithms. However, unlike unix pipes, this library attempts
> to make as few copies as possible of the data.
>
> example:
>
> foreach(line; (new IODevice(0)).bufferedInput
>      .asText!(UTFType.UTF8)
>      .byLine
>      .asInputRange)
>     // handle line
>
> - It can handle 5 forms of UTF encoding - UTF8, UTF16, UTF16LE, UTF32,
> UTF32LE (phobos only partially handles UTF8). Sorry, no grapheme support
> or other utf-related things, but this of course can be added later.
>
> - Arrays are first-class ioPipe types. This works:
>
> foreach(line; "one\ntwo\nthree\nfour\n".byLine.asInputRange)
>
> - Everything is compile-time for the most part, and uses lots of
> introspection. The intent is to give the compiler full gamut of
> optimization capabilities.
>
> - I added rudimentary compression/decompression support using
> etc.c.zlib. Using compression is done like so:
>
> foreach(line; (new IODevice(0)).bufferedInput
>      .unzip
>      .asText!(UTFType.UTF8)
>      .byLine
>      .asInputRange)
>
> - The plan is for this to be a basis to make super-fast and modular
> parsing libraries. I plan to write a JSON one as a proof of concept. So
> all you have to do is add a parseJSON function to the end of any chain,
> as long as the the input is some pipe of text data (including a string
> literal).
>
>
> =================
>
> I will stress some very very important things:
>
> 1. This library is FAR from finished. Even the concepts probably need
> some tweaking. But I'm very happy with the current API/usage.
>
> 2. Docs are very thin. Unit tests are sparse (but do pass).
>
> 3. The focus of this library is NOT replacement of std.stream, or even
> low-level i/o in general. In fact, I have copied over my stream class
> from previous attempts at this i/o rewrite ONLY as a mechanism to have
> something that can read/write from file descriptors with the right API
> (located in iopipe/stream.d). I admit to never having looked at
> std.stream really, so I have no idea how it would compare.
>
> 4. As the stream framework is only for playing with the other useful
> parts of the library, I only wrote it for my OS (OSX), so you won't be
> able to play out of the box on Windows (probably can be added without
> much effort, or use another stream library such as this one that was
> recently announced:
> https://forum.dlang.org/post/xtxiuxcmewxnhseubyik@forum.dlang.org), but
> it will likely work on other Unixen.
>
> 5. This is NOT thread-aware out of the box.
>
> 6. There is a concept in here I called "valves". It's very weird, but it
> allows unifying input and output into one seamless chain. In fact, I
> can't think of how I could have done output in this regime without them.
> See the convert example application for details on how it is used.
>
> 7. I expect to be changing the buffer API, as I think perhaps I have the
> wrong abstraction for buffers. However, I did attempt to have a
> std.allocator version of the buffer.
>
> 8. It's not on code.dlang.org yet. I'll work on this.
>
> Destroy!
>
> -Steve

A few things: 
https://github.com/schveiguy/iopipe/blob/master/source/iopipe/traits.d#L126 
why isn't that used more especially with e.g. window?
After all, window seems like a very well used word...

I don't like that a stream isn't inherently an input range.
This seems to me like a good place to use this abstraction by default.