Helping with __mutable (which will be renamed to __metadata)

[RazvanN]

On Saturday, 13 April 2019 at 00:45:07 UTC, Andrei Alexandrescu 
wrote:
> Razvan Nitu is working on the DIP initiated by Timon Gehr, 
> known colloquially as the one that introduces __mutable - i.e. 
> a mechanism for allowing controlled changes to immutable data. 
> Here's a draft:
>
> https://github.com/RazvanN7/DIPs/blob/Mutable_Dip/DIPs/DIP1xxx-rn.md
>
> We figured that we're dealing a misnomer - we don't want 
> __mutable, but instead __metadata - information that is 
> nominally part of the object but needs certain leeway from the 
> type system. Typical use cases are:
>
> * reference counting of immutable data structures
> * caching
> * lazy evaluation
>
> We got stuck at the interaction of __mutable with const parent 
> objects (unclear whether the parent object originated as 
> immutable or unqualified), and how pure functions should deal 
> with __mutable. The few solutions we are toying with are either 
> incomplete or too complicated (or both).
>
> The help of a few PL and compiler specialists would be very 
> valuable here. I'm cc'ing a few, if anyone wants to help please 
> let us know.

This is a draft proposal and it is not finished. The initial 
draft that was done by Timon [1] considers __mutable functions 
that act as optimization blockers for
the compiler with regards to `pure` functions. I feel that 
optimization blockers
for `pure` are a totally different concept than 
__mutable/__metadata fields.

Considering that the currently the compiler does not do any 
optimizations based on purity, I think it would be a lot easier 
to just get __mutable/__metadata fields
in, because they are a needed feature; after that, we can think 
on what optimizations we want to implement and how they interact 
with __mutable/__mutable.

Consider this code:

int foo() pure
{
      immutable(T)* x = allocate();
      int y = bar(x);
      deallocate(x);
      return y;
}

Currently, the compiler does not do any optimizations regarding 
purity with it. If it did, it could have swapped the invocation 
of `deallocate` with the one of `bar` leading to use-after-free. 
Annotating `deallocate` with __mutable would solve the issue in 
the event of compiler optimizations. However, this example has 
nothing to do with __mutable/__metadata fields; this kind of 
optimizations should be discussed in a DIP of their own and not 
in a __mutable/__metadata DIP. Moreover,
is this the direction we want to head in? Require the user to 
mentally trace the optimizations that the compiler might do? This 
is just too complicated.

That is why, I think that we should focus on implementing 
__metadata with regards to fields and later on think about the 
optimizations that we can perform.

As for __metadata fields:

1. If we decide that __metadata fields are not conceptually part 
of the
object, why would accesses to them be unsafe? We could  still 
make them `private`,
but we can view them as normal accesses from a @safety 
perspective.

2. I agree that `purity` should not be affected by 
__mutable/__metadata fields; the object passed as argument will 
not be conceptually modified.

If we take out optimizations out of the picture, things become a 
lot more clearer.
For me, the DIP is more of "how can we mutate fields in a 
non-mutable object", not
"how do we implement optimizations regarding purity".

I don't see why the addition of __metadata should be delayed by 
how optimizations based on purity are applied in the compiler,

Cheers,
RazvanN

[1] 
https://github.com/RazvanN7/DIPs/blob/Mutable_Dip/DIPs/timon_dip.md