An important pull request: accessing shared affix for immutable data

[ZombineDev via Digitalmars-d]

On Friday, 12 February 2016 at 19:12:48 UTC, Andrei Alexandrescu 
wrote:
> https://github.com/D-Programming-Language/phobos/pull/3991
>
> A short while ago Dicebot discussed the notion of using the 
> allocator to store the reference count of objects (and 
> generally metadata). The allocator seems to be a good place 
> because in a way it's a source of "ground truth" - no matter 
> how data is qualified, it originated as untyped mutable bytes 
> from the allocator.

I agree. Your allocators framework is very flexible and 
AffixAllocator looks like the perfect tool for the job. The type 
system already relies on the allocators and putting more trust in 
them can indeed enable some interesting opportunities.

> So after thinking a bit I managed to convince myself that the 
> affixes in an AffixAllocator can be accessed with removing 
> immutable, but without actually breaking any guarantee made by 
> the type system. (Affixes are extra bytes allocated before and 
> after the actual allocation.) The logic goes as follows:
>
> * If the buffer is mutable, then the allocator assumes it 
> hasn't been shared across threads, so it returns a reference to 
> a mutable affix.

I agree that non-shared data implies that it's meta data can't be 
shared, but I don't exactly agree with you reasoning - see below.

> * If the buffer is const, then the allocator must 
> conservatively assume it might have been immutable and 
> subsequently shared among threads. Therefore, several threads 
> may request the affix of the same buffer simultaneously. So it 
> returns a reference to a shared affix.

Absolutely disagree. See below.

> * If the buffer is shared, then the allocator assumes again 
> several threads may access the affix so it returns a reference 
> to a shared affix.

Correct. Shared data implies shared access to the corresponding 
metadata.

> One simple way to look at this is: the allocator keeps an 
> associative array mapping allocated buffers to metadata (e.g. 
> reference counts). The allocated buffers may be immutable, 
> which doesn't require the metadata to be immutable as well. The 
> only difference between an approach based on an associative 
> array and AffixAllocator is that the latter is faster (the 
> association is fixed by layout).
>

Yes, mutable metadata in front of immutable data is OK, as long 
as the users of the immutable data can't directly access the 
metadata (e.g. it is well encapsulated and those who do access 
it, must see is typed as `shared`).

> Destroy!
>
> Andrei

Overall it's an interesting and strong propsal, except for the 
part about prefix(const) magically returning shared.
Shared is transitive and unless you take special care to remove 
it - e.g.
http://dpaste.dzfl.pl/4294b9c2653e, you shouldn't be able to get 
non-shared references in to it (and vice-versa - you should have 
shared references in a non-shared objects).
That's why I believe that unless the allocator is explicitly 
shared, you shouldn't be allowed to share the bytes allocated 
from it.

So in order to get shareable immutable data, we should use shared 
Allocator and not try to subvert the type system, even though 
allocators are low-level infrastructure code.