std.container.BinaryHeap + refCounted = WTF???

[Steven Schveighoffer]

On Mon, 29 Nov 2010 11:44:23 -0500, dsimcha <dsimcha at yahoo.com> wrote:

> == Quote from Steven Schveighoffer (schveiguy at yahoo.com)'s article
>> On Mon, 29 Nov 2010 10:58:05 -0500, dsimcha <dsimcha at yahoo.com> wrote:
>> > == Quote from Steven Schveighoffer (schveiguy at yahoo.com)'s article
>> >> On Wed, 17 Nov 2010 12:09:11 -0500, dsimcha <dsimcha at yahoo.com>  
>> wrote:
>> >> > == Quote from Steven Schveighoffer (schveiguy at yahoo.com)'s article
>> >> >> The issue is that if you append to such an array and it adds more
>> >> pages
>> >> >> in
>> >> >> place, the block length location will move.  Since each thread  
>> caches
>> >> >> its
>> >> >> own copy of the block info, one will be wrong and look at array  
>> data
>> >> >> thinking it's a length field.
>> >> >> Even if you surround the appends with a lock, it will still cause
>> >> >> problems
>> >> >> because of the cache.  I'm not sure there's any way to reliably
>> >> append
>> >> >> to
>> >> >> such data from multiple threads.
>> >> >> -Steve
>> >> >
>> >> > Would assumeSafeAppend() do the trick?
>> >> >
>> >> No, that does not affect your cache.  I probably should add a  
>> function
>> >> to
>> >> append without using the cache.
>> >> -Steve
>> >
>> > How about using std.array.Appender?  This looks safe as far as I can
>> > tell, but I
>> > want to make sure there aren't any obscure implementation details that
>> > would
>> > prevent this from working, too.
>> That should be fine, std.array.Appender does not affect the LRU cache.   
>> In
>> fact, if you try to do normal appending to an array built with Appender,
>> it will automatically reallocate because the memory does not have the
>> APPENDABLE bit set (new GC bit I added to avoid misinterpretations).
>> This is a good solution.
>> -Steve
>
> Sounds like a good solution to me, too.  As long as this situation is  
> unlikely to
> change in the future, I think we should scrap the idea of making  
> appending to
> __gshared using ~= safe with manual synchronization,

I think it's worth giving people who want low-level access to the runtime  
functions the ability to avoid using the LRU cache.  The cache is meant to  
support everyday coding, but can possibly cause problems if you are  
interested in doing low-level optimizations.

> BUT the following needs to be
> documented:
>
> 1.  Due to obscure implementation details, it is **not** safe to append  
> to a
> non-shared array even if you synchronize manually.

I think you meant __gshared, not non-shared.  It's perfectly safe to  
append to non-shared arrays.

-Steve