[D-runtime] Precise garbage collection

[Steven Schveighoffer]

On Jun 23, 2013, at 8:51 AM, Rainer Schuetze <r.sagitario at gmx.de> wrote:

> On 23.06.2013 04:07, Jonathan M Davis wrote:
>> On Saturday, June 22, 2013 18:48:46 Steven Schveighoffer wrote:
>>> If you cast away shared, but don't ensure the data becomes actually
>>> unshared, you are in bad territory too.
>> 
> [...]
>> At this point, I'd be very nervous about anything in the runtime assuming that
>> a non-shared object won't be used across threads. It should be able to assume
>> that it's not _currently_ being used across threads (since it _is_ up to the
>> programmer to ensure that they don't violate the type system by having
>> multiple threads operate on a non-shared object at the same time), but in the
>> general case, I would not expect it to be able to assume that it won't be used
>> across threads in the future without being wrong.
> 
> I feel a little uneasy about that aswell, but I haven't really digged very deep into the implementation yet. Here's an example of a string shared between two threads, but no "shared" type around:
> 
> [...]
> 
> Which shows that the first thread appended in place, and the second reallocated. This is correct, but if both array append operations are executed concurrently, are the problems not the same as with "shared" strings?

As you guessed, this is not handled correctly by the runtime, which only expects shared data to be shared.  It is a quirk in the fact that array blocks with immutable data are not immutable where the data hasn't yet been written.  In other words, the runtime allows modification of the block via an immutable pointer.  It works fine for thread-local immutable data, but not for shared immutable data.

I don't have a good answer for this.  Your code will eventually result in a race if you ran it enough times.  At this point, I think I have to make immutable appends be like shared appends, not using the cache, and taking a lock.  The only other option I can think of is to make immutable appends always copy.  Neither of these solutions is attractive, and would hurt performance severely on code that doesn't share immutable data (the most common case).

Even worse, const array data could be immutable, could be shared, same problem.  So only mutable data would be able to be appended to without performance penalties.

Here is a possibility for rectifying: any time you share immutable data by reference, the compiler generates a call to some 'makeImmutableArrayShared' function with the pointer, and that removes the block's APPENDABLE bit (if part of the GC).  That would make any immutable shared arrays truly immutable, and avoid this problem.

Thoughts?  Is it possible to intercept the moment immutable data becomes shared?

-Steve