std.container.BinaryHeap + refCounted = WTF???

[Michel Fortin]

On 2010-11-16 15:47:07 -0500, dsimcha <dsimcha at yahoo.com> said:

> I'm trying to use BinaryHeap in a multithreaded programming using
> std.parallelism/parallelfuture.  I kept getting ridiculous segfaults and
> stuff, and when I looked into it in a debugger, I realized the crashes had to
> do with reference counting, probably caused by this line in BinaryHeap:
> 
>     private RefCounted!(Tuple!(Store, "_store", size_t, "_length"),
>                        RefCountedAutoInitialize.no) _payload;
> 
> Why the heck are the payload and the length being stored in such a weird way?
>  IMHO BinaryHeap shouldn't use reference counting unless Store does.  More
> generally, anything that uses reference counting, especially where unexpected,
> should come with a very strong warning in its ddoc so that people are aware of
> the hidden caveats, like copying it using memcpy() and sharing it 
> across threads.

RefCounted, and many other things in Phobos, aren't thread-safe because 
of those reference counters. This problem can occur even if you don't 
share it with other threads, as the GC may destroy objects in another 
thread. See bug 4624:
<http://d.puremagic.com/issues/show_bug.cgi?id=4624>

Also bug 4621 about the compiler not catching the problem at 
compile-time (as it should catch low-level races):
<http://d.puremagic.com/issues/show_bug.cgi?id=4621>

I suggest you try to make RefCounted use an atomic reference counter to 
see if it removes your segfaults.

Andrei acknowledged the issue recently (Walter too, I think), so I 
trust it'll be fixed at some point. One idea was to change the GC so it 
knows in which thread it should call destructors...

-- 
Michel Fortin
michel.fortin at michelf.com
http://michelf.com/