Rebooting the __metada/__mutable discussion

[Timon Gehr]

On 07.04.22 10:51, RazvanN wrote:
> On Wednesday, 6 April 2022 at 18:03:25 UTC, Timon Gehr wrote:
>> On 06.04.22 11:41, RazvanN wrote:
>>> As such, I would propose that `__metadata` can only be accessed from 
>>> inside
>>> the aggregate that defines it (`private` here means `Java private`) 
>>> and methods
>>> that access __metadata directly need to also private.
>>
>> Those rules are pretty arbitrary and don't buy much. However, 
>> accessing `__metadata` should be `@system`.
>>
> They buy the fact that __metadata fields cannot be accessed from outside
> of the object that implements the reference count.

Not really, e.g., just expose a pointer to it. I really don't see why we 
need special-case visibility rules just for this.

Note that reference counts are not the only use case. You can also do 
e.g., lazy initialization.

> This offers the 
> guarantee
> that a strongly pure function will not alter a __metadata field

Not really.

> and hence it
> can be the subject of any purity-based optimization

That's backwards. The purity-based optimizations determine what you are 
allowed to do to `__metadata` fields. Because you shouldn't do arbitrary 
stuff there it should be `@system`, as it's up to the programmer to 
uphold guarantees here, not the compiler. It's a low-level feature.

> (with the exception of functions that do deallocations).
> ...

Note that this means you have to mark functions that are used from 
destructors specially.

>>>
>>> What do you think? Am I missing anything?
>>
>> E.g., how to deallocate memory in a pure destructor? (Solved by 
>> `__mutable` functions in my original draft.)
>>
>> What if the destructor is both pure and immutable?
>>
> 
> Destructors suffer from the same issue as postblits did with regards to 
> qualifiers. I would be surprised if you could ever call an immutable 
> destructor.

Well, what I meant is what happens if you destruct an immutable object. 
There need to be some language rules that allow you to have memory that 
is only temporarily immutable, otherwise your reference counting scheme 
will never work for immutable memory anyway.

> Combined with the fact that you cannot overload the 
> destructor makes everything worse. So I would argue that talking about 
> immutable pure destructors in this context is like talking about a 
> broken glass when you're in a house on fire.
> 
> However, I do get your point and in order to support immutable 
> allocators we do need to take into consideration how allocation and 
> deallocation is done.
> 
> I think that the problem here is that we are conflating a high-level 
> concept (purity) with low level operations (allocation/deallocation). 

It's not really a conflation, it's that you implement the high-level 
concept in terms of low-level operations. This is not problematic, this 
is how it always works.

> The latter category is essentially not pure because it modifies 
> operating system data structures, so I would say that we could view them 
> as system code, not in the sense that they are unsafe (although they 
> could be), but because they require OS assistance.

They are unsafe because you need them to behave in a certain way for it 
to be possible to consider them `pure`.

> Now, this sort of 
> functions should be at most weakly pure (and this is what mutable 
> functions essentially did) even though their signature looks as if they 
> were strongly pure.

I don't think this has much to do with weakly vs strongly pure. It's a 
different category.

> One solution would be to consider @system/@trusted 
> pure functions as weakly pure no matter what their signature looks like. 
> Constructors/destructors should also be considered at most weakly pure. 
> If your function is @safe and pure then the signature should be analyzed 
> to decide if it is strongly or weakly pure. Also, any function that you 
> are calling from another language should be at most trusted. This way, 
> @trusted would act as an optimization blocker.
> ...

There is no good reason whatsoever why @trusted or @system should block 
optimizations. Orthogonal language design, please. :(