[RFC] I/O and Buffer Range
Jason White
54f9byee3t32 at gmail.com
Mon Jan 6 23:59:59 PST 2014
On Monday, 6 January 2014 at 10:26:27 UTC, Dmitry Olshansky wrote:
> Ok, now I see. In my eye though serialization completely hides
> raw stream write.
>
> So:
> struct SomeStream{
> void write(const(ubyte)[] data...);
> }
>
> struct Serializer(Stream){
> void write(T)(T value); //calls stream.write inside of it
> private:
> Stream stream;
> }
I was thinking it should also have "alias stream this;", but
maybe that's not the best thing to do for a serializer.
I concede, I've s/(read|write)Data/\1/g on
https://github.com/jasonwhite/io/blob/master/src/io/file.d
and it now works on Windows with useful exception messages.
>> Shouldn't buffers/arrays provide a stream interface in
>> addition to
>> buffer-specific operations?
>
> I think it may be best not to. Buffer builds on top of
> unbuffered stream. If there is a need to do large reads it may
> as well use naked stream and not worry about extra copying
> happening in the buffer layer.
>
> I need to think on this. Seeing as lookahead + seek could be
> labeled as read even though it's not.
>
>> I don't see why it would conflict with a
>> range interface. As I understand it, ranges act on a single
>> element at a
>> time while streams act on multiple elements at a time. For
>> ArrayBuffer
>> in datapicked, a stream-style read is just lookahead(n) and
>> cur += n.
>> What capabilities are lost?
>
> In short - lookahead is slicing, read would be copying.
> For me prime capability of an array is slicing that is dirt
> cheap O(1). On the other hand stream interface is all about
> copying bytes to the user provided array.
>
> In this setting it means that if you want to wrap array as
> stream, then it must follow generic stream interface. The
> latter cannot and should not think of slicing and the like.
> Then while wrapping it in some adapter up the chain it's no
> longer seen as array (because adapter is generic too and is
> made for streams). This is what I call capability loss.
>
>> If buffers/arrays provide a stream interface, then they can be
>> used by
>> code that doesn't directly need the buffering capabilities but
>> would
>> still benefit from them.
>
> See above - it would be better if the code was written for
> ranges not streams. Then e.g. slicing of buffer range on top of
> array works just as cheap as it was for arrays. And zero copies
> are made (=performance).
Okay, I see. I'm just concerned about composability. I'll have to
think more about how it's affected.
(BTW, you can probably simplify lookahead/lookbehind with
look(ptrdiff_t n) where the sign of n indicates ahead/behind.)
> Actually these objects do just fine, since OS does the locking
> (or makes sure of something equivalent). If your stream is TLS
> there is no need for extra locking at all (no interleaving of
> I/O calls is possible) regardless of its kind.
>
> Shared instances would need locking as 2 threads may request
> some operation, and as OS locks only on per sys-call basis
> something cruel may happen in the code that deals with
> buffering etc.
Oh yeah, you're right.
As a side note: I would love to get a kick-ass I/O stream package
into Phobos. It could replace std.stream as well as std.stdio.
Stuff like serializers and lexers would be more robust and easier
to write.
More information about the Digitalmars-d
mailing list