Suggestions for std.stream (esp. for 64bit)

[Daniel Gibson]

I'm using some of std.stream's classes (and also SocketStream) in a 
project of mine (it currently uses D1 but std.Stream hasn't changed much 
so all this is valid for D2 as well).
std.stream is mostly what I expect from classes for Streams (having 
experience with Java's equivalents). However there are a few things that 
I think could/should be improved.

1. (important for 64bit D): write(char[]) and read(char[]) are too 
platform specific.
These methods (and the wchar counterparts) write (or read) "a string, 
together with its length". This is fine (even though I would rather 
expect writeString/readString to do this magic, but nevermind), *but* 
the length is written as a size_t - making it heavily platform dependant.
This means that you can't use write(char[]) to write into a file on a 
x86 system and later read that file on an amd64 system. Also consider 
SocketStream.. you can't use SocketStream.write(char[]) to communicate 
between a x86 and an amd64 box (when an 64bit executable is used an the 
latter).
This could easily be fixed by using uint or ulong instead of size_t on 
all platforms. (uint is probably ok, Java even uses short in a similar 
method (java.io.DataOutput.writeUTF() - never use this, it's no real 
UTF-8)).

Unfortunately the libphobos of GDC (that already supports 64bit targets) 
has been using size_t for ages, so in D1 it should maybe stay like that 
to avoid breaking compatibility (on the other hand probably no GDC user 
who thinks at least a bit cross-platform uses write(char[]) anyway - in 
that case just use uint so it's compatible with existing 32bit binaries 
from DMD).
But at least for D2/phobos2 that should be changed.

2. The documentation says for write(): "Outside of byte, ubyte, and 
char, the format is implementation-specific and should only be used in 
conjunction with read. Throw WriteException on error."
So how do I write files for other programs to read? How do I communicate 
with servers/clients not written in D with a SocketStream?
Fortunately the documentation exaggerates. Apart from write(char[]) and 
write(wchar[]) the claim is not true.
The simple types are platform specific, not specific to D/std.stream - 
so a program written in C (or any other language) is probably able to 
read that data (if it supports the type - may be tricky with real and 
maybe the imaginary types).
And by using the EndianStream, most of the types probably can be read by 
other platforms (again, real and the imaginary types might cause 
trouble.. and maybe floating point types in general, if the other 
platform doesn't support IEEE 754 floats - but integer types are 
definitely safe).

This is bad, because if someone wants to use SocketStream he's confused 
by that statement until he looks at the source to find out that it's not 
so implementation specific after all.
So please document how exactly write( (w)char[]) encodes the length and 
also make clear that write( <basic type> ) does no strange voodoo, but 
just dumps the bytes of the value (in platform specific big/little 
endian order).

3. InputStream's read( <type> val) often is inconvenient.
If you want to read an int (or any other basic type) from an InputStream 
s you have to do:
   int foo; s.read(foo);
This doesn't look soo horrible.. but maybe you want to pass that value 
directly to a function?
   int foo; s.read(foo); bar(foo);
This is inconvenient. Java has something like Stream.readInt(), so you 
can write
   bar( s.readInt() );
That's much shorter and you don't need to invent a name for that value 
you only want to use once anyway.
For D I'd suggest a templated method for that:
   T read(T)() { T ret; readExact(&ret, ret.sizeof); return ret; }
So you could write
   int foo = s.read!int; // instead of int foo; s.read(foo);
or even
   auto foo = s.read!int;
and
   bar( s.read!int ); // instead of "int foo; s.read(foo); bar(foo);"
(The implementation should probably make sure T is a basic type or maybe 
a struct, but no array or Object or pointer. Also special cases for 
char[] and wchar[] might be needed for consistency.)

4. Minor inconsistencies:
In InputStream there is a read(ubyte[]) method, but no 
readExact(ubyte[]) method.
I'd suggest adding
   void readExact(ubyte[] buf) { readExact(buf.ptr, buf.length); }
and, for convenience
   ubyte[] readExact(size_t len) {
     ubyte[] ret = new ubyte[len];
     readExact(ret.ptr, len);
     return ret;
   }

Also there is a write(ubyte[]) method in OutputStream, but no 
writeExact(ubyte[]) method, so I'd suggest adding
   void writeExact(ubyte[] buf) { // maybe "const(ubyte[]) buf" for D2
     writeExact(buf.ptr, buf.length);
   }



All code above is untested and thus to be considered pseudo-code ;-)

Cheers,
- Daniel