Helping with __mutable (which will be renamed to __metadata)

[RazvanN]

On Wednesday, 17 April 2019 at 01:00:35 UTC, Timon Gehr wrote:
> On the contrary, it's the type system information you _need_ in 
> order to abstract over code that mutates __metadata. An 
> assignment to a __metadata field is something you cannot ignore 
> even though it is `pure` and does not return any interesting 
> value. For any statement, you need to be able to assign a valid 
> type to a function that contains this statement, and that 
> captures what the type system knows about the statement.
>
> You need to specify that there are optimizations that are 
> allowed even though they change the set of mutations that the 
> program performs. As I suggested earlier, if you want the 
> semantics that's least restrictive for language evolution, you 
> need to say something that implies that all optimizations based 
> on the function signature are allowed, not "the compiler 
> currently does not do any optimizations". That's precisely the 
> wrong direction if you want to be conservative.
>
I see, now I understand.
>
> What the compiler does and does not do has no bearing on the 
> validity of the code per language semantics.
>
> That's simply nonsense. You define the language semantics, the 
> compiler developer figures out optimizations that are 
> consistent with that semantics. You don't need any DIPs for 
> optimizations, you need DIPs to specify how programs may or may 
> not behave. You can't really first come and say programs may 
> not behave a certain way and then later relax that and say that 
> actually those programs can also behave in this other way and 
> at the same time claim that this is a conservative approach. It 
> is not; it breaks code.
>
I understand now. It was very confusing for me that `pure` was
simply implemented as a set of compile time checks not backed by
any optimizations. I figured that if none of them were implemented
by now, it will certainly break a lot of code when we will have 
them,
so there is a great chance that `pure` will simply remain in the
current stage. If that would be the case, adding the cognitive 
load
for __metadata functions to the user is not worth it.
>
>> Moreover,
>> is this the direction we want to head in?
> Absolutely. If you want to use __metadata you need to be aware 
> that pure functions can be elided, and if the mutation of 
> __metadata within a function is not self-contained such that it 
> can be elided, you need to annotate that function.
>
>> Require the user to mentally trace the optimizations that the 
>> compiler might do?
>
> The user of __metadata. _Some_ kinds of optimizations.
>

But it's not just the user of __metadata. If you implement in a 
library a refcounted object then __metadata is going to be 
propagated to user code.

>> This is just too complicated.
>> ...
>
> Then give up on __metadata and just implement your persistent 
> data structures without the `immutable` qualifier slapped on 
> top! `immutable` has no value if it cannot be used to justify 
> equational reasoning on pure functions.
>
>> That is why, I think that we should focus on implementing 
>> __metadata with regards to fields and later on think about the 
>> optimizations that we can perform.
>> 
>> As for __metadata fields:
>> 
>> 1. If we decide that __metadata fields are not conceptually 
>> part of the
>> object, why would accesses to them be unsafe? We could  still 
>> make them `private`,
>> but we can view them as normal accesses from a @safety 
>> perspective.
>> ...
>
> Then those accesses can't be pure because they are like 
> accesses to global variables.

But they actually aren't because they are "physically" part of 
the object.

>
>> 2. I agree that `purity` should not be affected by 
>> __mutable/__metadata fields; the object passed as argument 
>> will not be conceptually modified.
>> ...
>
> 1 and 2 are fundamentally at odds with each other if you want 
> __mutable fields to be accessible in pure functions, because 
> you can't check 2 safely.
>
>
>> If we take out optimizations out of the picture, things become 
>> a lot more clearer.
>
> You need to define the semantics!

What I was trying to say is that it would be easier to define 
semantics
if we wouldn't pretend that the compiler treats `pure` in a way 
that it
actually doesn't. I see your point, however, even if we have 
__metadata/
__mutable functions, once the purity optimizations will be 
implemented,
all the users will have to check their code to add 
__metadata/__mutable
to their functions to be sure their calls to strongly pure 
functions are
not elided.

>
>> For me, the DIP is more of "how can we mutate fields in a 
>> non-mutable object", not
>
> You clearly can't mutate fields in a non-mutable object except 
> non-mutable is some higher-level abstraction. That abstraction 
> has to be defined. One way to define semantics for __metadata 
> is to explicitly give the set of allowed rewrites, but you can 
> also define it implicitly.
>
>> "how do we implement optimizations regarding purity".
>> 
>> I don't see why the addition of __metadata should be delayed 
>> by how optimizations based on purity are applied in the 
>> compiler,
>> 
>> ...
>
> It shouldn't.

 From an academical perspective, I agree with everything you said 
and I think
I finally understand your point of view and your DIP draft. I 
think that a
small chat would have made it easier for both parties to reach an 
understanding.

At this point, I think it all comes down to what we want to do 
with `pure`. It really doesn't make any sense to take all these 
optimizations into account and
burden the user with __mutable functions if those optimizations 
will never be
implemented.

Cheers,
RazvanN