Stream Desing (Was: GZip File Reading)

[Jonathan M Davis]

On Thursday, March 10, 2011 09:16:35 dsimcha wrote:
> == Quote from Jonathan M Davis (jmdavisProg at gmx.com)'s article
> 
> > There _have_ been some threads on designing a new stream API, and there
> > are some preliminary designs, but as far as I know, they have yet to
> > really go anywhere. I'm unaware of the stream API really having a
> > champion per se. Andrei has done some preliminary design work, but I
> > don't know if he intends to actually implement anything, and as far as I
> > know, no one else has volunteered. So, a new std.stream is one of those
> > things that we all agree on that we want but which hasn't happened yet,
> > because no one has stepped up to do it.
> > And I do agree that the ability to deal with a compressed file should be
> > part of the stream API (probably as some sort of adapter/wrapper which
> > uncompresses the stream as you iterate through it). But the stream API
> > needs to be designed and _implemented_ before we'll have that.
> > - Jonathan M Davis
> 
> So I guess in such a design we would still have things like a decorator to
> iterate through a stream by chunk, by line, etc., with a range interface?

I don't remember exactly what it's going to look like. IIRC, streams are the 
reason that Andrei was looking at at a range API that gave you a T[] instead of 
T. Whatever you were doing would be built on top of that. So, grabbing it by 
line or whatever. But I would fully expect that you would be able to put a 
wrapper range in there which took the stream of bytes, treated them as if they 
were gzipped or bzipped or whatever, and gave you bytes (or chars or whatever 
was appropriate) as if were reading from an uncompressed version of the file. So, 
the reading would be identical whether the file was compressed or not. It's just 
that in the case of a compressed stream/file, you'd have decorator/wrapper which 
uncompressed it for you.

- Jonathan M Davis