buffered input

[Jean Crystof]

Tomek Sowiñski Wrote:

> Andrei Alexandrescu napisa³:
> 
> > I hereby suggest we define "buffered input range of T" any range R that 
> > satisfies the following conditions:
> > 
> > 1. R is an input range of T[]
> > 
> > 2. R defines a primitive shiftFront(size_t n). The semantics of the 
> > primitive is that, if r.front.length >= n, then shiftFront(n) discards 
> > the first n elements in r.front. Subsequently r.front will return a 
> > slice of the remaining elements.
> > 
> > 3. R defines a primitive appendToFront(size_t n). Semantics: adds at 
> > most n more elements from the underlying stream and makes them available 
> > in addition to whatever was in front. For example if r.front.length was 
> > 1024, after the call r.appendToFront(512) will have r.front have length 
> > 1536 of which the first 1024 will be the old front and the rest will be 
> > newly-read elements (assuming that the stream had enough data). If n = 
> > 0, this instructs the stream to add any number of elements at its own 
> > discretion.
> 
> I don't see a clear need for the two to be separate. Could they fold into popFront(n, m) meaning shiftFront(n); appendToFront(m) ? Nullary popFront() discards all and loads any number it pleases.
> 
> > This is it. I like many things about this design, although I still fear 
> > some fatal flaw may be found with it.
> >
> > With these primitives a lot of good operating operating on buffered 
> > streams can be written efficiently. The range is allowed to reuse data 
> > in its buffers (unless that would contradict language invariants, e.g. 
> > if T is invariant), so if client code wants to stash away parts of the 
> > input, it needs to make a copy.
> 
> Some users would benefit if they could just pass in a buffer and say "fill'er up".
> 
> > One great thing is that buffered ranges as defined above play very well 
> > with both ranges and built-in arrays - two quintessential parts of D. I 
> > look at this and say, "this all makes sense". For example the design 
> > could be generalized to operate on some random-access range other than 
> > the built-in array, but then I'm thinking, unless some advantage comes 
> > about, why not giving T[] a little special status? Probably everyone 
> > thinks of contiguous memory when thinking "buffers", so here 
> > generalization may be excessive (albeit meaningful).
> 
> Contiguous, yes. But I'd rather see front() exposing, say, a circular buffer so that appendToFront(n) reallocates only when n > buf.length.

I find this discussion interesting. There's one idea for an application I'd like to try at some point. Basically a facebook chat thingie, but with richer gaming features. The expected audience will be 10 - 100K simultaneous clients connecting to a single server. Not sure if DOM or SAX will be better. After seeing the Tango's XML benchmarks I was convinced that the implementation platform will be D1/Tango, but now it looks like Phobos is also getting there, propably even outperforming Tango by a clear margin.

Since even looking at Tango's documentation has intellectual property problems and likely causes taint, I could make an independent benchmark comparing the two and their interfaces later. But I propaply need to avoid going into too much details, otherwise the Phobos developers wouldn't be able to read it without changing their license. From what I've read so far, the proposed design looks very much like what Tango has now in their I/O framework. But probably Phobos's TLS default and immutable strings improve multithreaded performance even more.