streaming redux

[Daniel Gibson]

Am 28.12.2010 08:02, schrieb Andrei Alexandrescu:
> I've put together over the past days an embryonic streaming interface.
> It separates transport from formatting, input from output, and buffered
> from unbuffered operation.
>
> http://erdani.com/d/phobos/std_stream2.html
>
> There are a number of questions interspersed. It would be great to start
> a discussion using that design as a baseline. Please voice any related
> thoughts - thanks!
>
>
> Andrei

I think I mostly like the proposal. I think it should be done this way, 
i.e. in OOP style, not with ranges-like duck-typing.
Here are my comments:

## TransportBase:
 > Question: May we eliminate seekFromCurrent and  seekFromEnd and just
 > have seek with absolute positioning? I don't know of streams that
 > allow seek without allowing tell. Even if some stream doesn't, it's
 > easy to add support for tell in a wrapper. The marginal cost of
 > calling tell is small enough compared to the cost of  seek.

No, seekFromCurrent may be convenient e.g. in network streams (just skip 
some bytes), where other seek operations don't make that much sense.

 > Question: Should this [close()] throw on an unopened stream?

No, close()ing a closed stream should just do nothing.

## UnbufferedInputTransport:

I'd like "void readFully(ubyte[] buffer)" which reads buffer.length 
bytes or throws an exception if that is not possible
This would also fix busy-waiting (it'd block until buffer.length bytes 
are available).

Also "size_t read(void* buffer, size_t length)" (and the same for 
readFully()) would be nice, so one can read from the stream to buffers 
of arbitrary type without too much casting. Is probably especially handy 
when used with (data from) extern(C) functions and such.

Also, for convenience: "ubyte[] read(size_t length)" (does something 
like "return read(new ubyte[length]);"
and "ubyte[] readFully(size_t length)"

## UnbufferedOutputTransport:

I'd like "void write(void *buffer, size_t length)" - for the same reason 
as read(void* buffer, size_t length).

## Formatter:

 > Question: Should all formatters require buffered transport? Otherwise
 > they might need to keep their own buffering, which ends up being less
 > efficient with buffered transports.

No, I don't think so. But readFully() would come in handy for that case.

## Unformatter:

Why is "abstract void put(Object obj);" here and not in Formatter?

*Please* provide not only "void read(ref <PRIMITIVE_TYPE> value)" but 
also "<PRIMITIVE_TYPE> read<TYPENAME>()". I found this design in the old 
std.stream quite annoying.

Just compare:
int i;
tr.read(i);
foo(i);
with:
foo(tr.readInt());

Or maybe even only this alternative (I don't think you gain anything by 
passing a reference to a variable to read() instead of just returning 
the variable).


"abstract void read(ref char[] value);" etc:
 > Question: Should we pass the size in advance, or make the stream
 > responsible for inferring it?

reading a string without knowing its length doesn't make much sense in 
95% of the cases (assuming that 5% of the cases read only fixed length 
strings), so the user would have to make sure the length is prepended 
himself, so he knows how long "value[]" is.
So it may make sense to write the strings length in front of the string 
itself, as a uint or ulong or something (but *not* a size_t like in old 
std.stream, because that is not portable between i386 and amd64!).
Else one could just use "abstract void read(ref void[] value, TypeInfo 
elementType);" instead.

Cheers,
- Daniel