std.allocator issues

[Steven Schveighoffer via Digitalmars-d]

On 2/20/16 8:47 AM, Andrei Alexandrescu wrote:
> On 02/20/2016 12:39 AM, Steven Schveighoffer wrote:
>> Given that there is "goodAllocSize", this seems reasonable. But for
>> ease-of-use (and code efficiency), it may be more straightforward to
>> allow returning more data than requested (perhaps through another
>> interface function? allocateAtLeast?) and then wrap that with the other
>> allocation functions that simply slice the result down to size.
>
> Actually I confess that was the exact design for a while, and I, too,
> found it ingenious. You ask me for 100 bytes but I'll allocate 256
> anyway, why not I just return you the whole 256? But then that puts
> burden everywhere. Does the user need to remember 256 so they can pass
> me the whole buffer for freeing?

I hadn't considered that aspect, it's a good point. Some allocators may 
care what you pass for length on free.

Using goodAllocSize, you can probably create wrapper primitives for all 
the allocators that do what I want, so the building blocks you have are 
likely the right choice.

>
>> In the GC, expanding is done a page at a time. If I request an expansion
>> of 1 byte, that's 4095 wasted bytes. I don't expect goodAllocSize to
>> help me here.
>
> If you call goodAllocSize(n + 1) you should get the right thing. Then
> you can pass it to the call to expand().

Expand takes the delta, so it would be goodAllocSize(n + 1) - n, but 
it's doable.

I found some bugs in GCAllocator.expand, will submit a PR.

>
>> BTW, just pushed an update to my i/o library that uses allocators
>> exclusively for buffers. But I make the default a custom GC allocator
>> that has the properties I need. I'm hoping there will be a way to get
>> the desired behavior from the phobos version at some point.
>
> Sounds great. Did you measure performance?

Performance hasn't gotten worse, if that's any comfort. I wouldn't 
expect the calls to Allocator to matter, since the library is built to 
minimize calls for allocation :)

The current byline performance beats Phobos, likely not c getline, but I 
do support UTF delimiters, whereas c getline and Phobos do not. I'd like 
to get the performance better, but I need to write a test for getline to 
see what I'm up against. The performance for converting UTF data is good 
(no way to compare to Phobos, since it only supports UTF8 streams), I 
think it beats my previous incarnations of the i/o library that did the 
same thing. My sample zip program is roughly the same performance as the 
gzip command-line command. I'm pretty happy with the performance for as 
little time as I spent tuning.

The zip pipes were pretty interesting, and I'm going to make some more 
building blocks to generalize the concept of copying from one buffer to 
another under the hood.

-Steve