Phobos for Review: std.buffer.scopebuffer

[Walter Bright]

On 2/7/2014 12:46 AM, Brad Anderson wrote:
> On Friday, 7 February 2014 at 08:24:39 UTC, Walter Bright wrote:
>> https://github.com/D-Programming-Language/phobos/pull/1911
>>
>> This adds a package std.buffer, and the first entry in that package,
>> std.buffer.scopebuffer.
>
> About to go to bed so just a couple of quick thoughts.
>
> Why not just stick the stack array in ScopeBuffer itself (the
> size could be a template parameter)?

1. It's set up to fit into two registers on 64 bit code. This means it can be 
passed/returned from functions in registers. When I used this in my own code, 
high speed was the top priority, and this made a difference.

2. It needs to avoid the default initialization of the array, because it's both
unnecessary and it kills the speed. Currently, this
cannot be avoided for part of a struct.

3. I wanted it to be usable for any block of memory the user wished to dedicate 
to be a buffer.

4. Having an internal reference like that would mean a postblit is required for 
copying/moving the range, another source of speed degradation.


The design of ScopeBuffer is based on a fair amount of trial and error on my 
part trying to wring as much speed as possible out of it.


> Also, if it did embed the static array in the ScopeBuffer I
> wonder if it could just use the pointer as the buffer if the
> length is less than size_t (short string optimization).

I almost never needed a buffer that small, and the extra tests for it killed 
performance (I had tried it at one point).


> (Just thinking aloud, I wonder if std.allocators could be used to
> implement this more generically (hypothetically: alias
> ScopeBuffer =
> CustomAllocOutputRange!(StaticWithMallocFallbackAllocator!10)))

The custom allocator design is a long way from being realized. I don't know if 
it would help or not.


> It's a welcome addition and will go a long way toward cutting
> down GC allocations once Phobos is output-range-ified. This technique is
> probably the first thing I'd reach for when choosing an output range.

Once the buffer package is accepted, I want to add others I've been using like 
circular buffers (the one from the compressor proposal), fixed buffers, and even 
move std.outbuffer to it. I also want it to be one buffer design per module, not 
the usual grab-bag modules.