Memory allocation in D

[Sean Kelly]

Jason House wrote:
> Sean Kelly wrote:
> 
>> Craig Black wrote:
>>> "Sean Kelly" <sean at f4.ca> wrote in message
>>> news:fin4fl$1h7v$1 at digitalmars.com...
>>>> This thread inspired me to make a number of changes to the GC and such
>>>> in
>>>> Tango.  They should improve efficiency in terms of eliminating wasted
>>>> space and avoiding collections in some situations.  They'll probably
>>>> take a few days, so stay tuned.
>>> Cool!  Maybe you could show some benchmarks when you get done?
>> Unfortunately, I think I'm not going to commit these changes.  I had
>> some clever ideas for how to replace the "allocate an extra byte"
>> functionality with an approach that would have resulted in less wasted
>> space, but it's become more complicated than the added efficiency is
>> worth.
>>
>> Basically, I was hoping to keep an extra free page at the end of the
>> allocated memory region used by the process.  This assumes that all
>> memory used by the process is contiguous however, or at least that the
>> intervening pages are valid.  But neither assumption is reasonable, and
>> while some sort of fancy manual fix may have worked with some memory
>> management APIs, it wouldn't have worked with others.
>>
>> For what it's worth, my concern with the current behavior is that its
>> worst-case scenario coincides with typical usage.  It is extremely
>> common for a programmer to use array sizes that are powers of two, and
>> this is the situation which results in the maximum unused space per
>> block (because the runtime adds 1 to the size of all array allocations
>> passed to the GC).  Totally stinks, and I'm running out of ideas.  I
>> don't suppose someone can offer a suggestion for how to address this,
>> other than "just always allocate exactly what they request and if they
>> get a page fault then too bad for them?"
> 
> If object sizes are a power of 2, is that because of special padding of a
> struct/object to make it align well?  Maybe some clever trick could be done
> to use wasted space, if it exists, for special tracking data... Otherwise,
> allocating a bit more seems the most sensible.

Only arrays get the +1 byte added to their size during allocations.  If 
an object occupies 16 bytes then it will end up in a 16 byte bucket. 
The +1 issue for arrays exists to make the program a bit more forgiving 
for "past the end" errors.  Interestingly, the ANSI C standard actually 
requires this behavior in its allocators.  I have yet to decide whether 
or not this is an artifact of C using terminators to mark the end of a 
block, since using terminators seems more likely to result in such bugs 
than the length scheme D employs.


Sean