std.container.BinaryHeap + refCounted = WTF???

[Steven Schveighoffer]

On Wed, 17 Nov 2010 09:17:05 -0500, dsimcha <dsimcha at yahoo.com> wrote:

> == Quote from Steven Schveighoffer (schveiguy at yahoo.com)'s article
>> I think in general containers don't work across multiple threads unless
>> specifically designed to do that.
>
> I'm making the assumption that you'd handle all the synchronization  
> issues
> yourself.  When you need to update the container, there's an obvious  
> issue.  In
> general I don't like the trend in D of building things into arrays,  
> containers,
> etc. such that they can't be shared across threads due to obscure  
> implementation
> details even when it looks safe.

I think that we need a wrapper for containers that implements the shared  
methods required and manually locks things in order to use them.  Then you  
apply this wrapper to any container type, and it's now a shared container.

There are also certain types of containers that lend themselves to shared  
access.  For example, I can see a linked list where each node contains a  
lock being a useful type.

> (For arrays, I'm referring to the appending issue, which is problematic  
> when I try
> to append to an array from multiple threads, synchronizing manually.)

I'm interested what you mean here, I tried to make sure cross-thread  
appending is possible.

>> dcollections containers would probably all fail if you tried to use them
>>  from multiple threads.
>> That being said, I'm not a huge fan of reference counting period.
>> Containers have no business being reference counted anyways, since their
>> resource is memory, and should be handled by the GC.  This doesn't mean
>> pieces of it shouldn't be reference counted or allocated via malloc or
>> whatever, but the object itself's lifetime should be managed by the GC
>> IMO.  Not coincidentally, this is how dcollections is set up.
>
> I think reference counting is an elegant solution to a niche problem  
> (namely,
> memory management of large containers that won't have circular  
> references when
> memory is tight), but given all the baggage it creates, I don't think it  
> should be
> the default for any container.  I think we need to start thinking about  
> custom
> allocators, and allow reference counting but make GC the default.

The memory management of a container's innards is open to reference  
counting (or whatever, I agree that allocators should be supported  
somewhere).  I just object to reference counting of the container itself,  
as it's not important to me whether a container gets closed outside the GC  
automatically.

With something like a File it's different since you are coupling closing a  
file (a very limited resource) with cleaning memory (a relatively abundant  
resource).

-Steve