OutputRanges and slicing/save()

[monarch_dodra]

On Friday, 3 January 2014 at 22:56:41 UTC, Justin Whear wrote:
> I've run into a design issue surrounding ranges and am looking 
> for advice
> on the best way to proceed.  To illustrate the issue, consider 
> the
> Shapefile format: a 100 byte header followed by variable-length 
> records.
> The tricky bit is that the header includes a field which 
> contains the
> total length of the file (as measured in 16-bit words, 
> curiously).  The
> header must be written first, but the total length of the file 
> isn't
> known until all the records have been encoded.  When writing to 
> a File
> this isn't a problem: write 100 bytes of padding, write the 
> records, use
> rewind(), and write the proper header.  It's in the context of 
> an
> OutputRange that I don't know how to proceed.  Consider the 
> most flexible
> range type: the array.  An array is not an OutputRange, so it 
> needs to be
> wrapped in something like std.array.Appender.

Actaully, dynamic arrays *are* output ranges. Each "put" places 
the element at the front of the range, and the range is then 
pop'ed front.

Its' not an "expanding" output range, rather, a "fillable" output 
range, so not really something that would fit your need.

> Ideally I could save off
> the initial state of the range, write a bogus header, write the 
> records,
> then jump back and write the proper header.  Unfortunately, 
> Appender is
> not a ForwardRange, nor does it appear that the field of 
> OutputRanges
> which are also ForwardRanges has been explored.  I'm using the 
> excellent
> read, write, and append functions from std.bitmanip, so write() 
> would fit
> the bill if only Appender supported slicing.
>
> My current solution is require the user to construct the 
> ShapeWriter
> output range (which supports `put(Shape)`) over two separate 
> output
> ranges of ubyte: one for the header and a second for the 
> records, then
> delay writing to the header range until the record range is 
> complete.
> This is both needlessly complex and it leaves the proper 
> combination of
> the two to the user, making ShapeWriter a very leaky 
> abstraction.
>
> So, for particular questions:
> 1) Am I missing something in Phobos that would provide an 
> OutputRange of
> ubytes while also providing ForwardRange/slicing capabilities?

<pedantic>ubyte[] is such a range</pedantic>. It won't do what 
you want though.

> 2) I know the relationship between streams and ranges has been 
> discussed
> at least once before; is the concept of rewinding and 
> overwriting simply
> incompatible with OutputRange in general?
>
> Justin

I think the biggest issue is that there is actaully 0 relation 
between input ranges and output ranges. The two concepts are 
completely orthogonal.

In particular, arguably, input ranges that are also output ranges 
are confusing,  since rather than "growing as you add items to 
their tip (what you'd want)", they instead "shrink as you 
overwrite their front, until they are empty/full".

When using put(range, item), the "put" primitive of "range" takes 
precedence over "front". You could design a range that uses that, 
which would look a bit more like what you want, but there is 
nothing that exists in phobos that does this that I know of 
anyways.

It's a bit of a mess (IMO).