Rebooting the __metada/__mutable discussion

[Timon Gehr]

On 4/9/22 02:56, Zach Tollen wrote:
> On Friday, 8 April 2022 at 17:28:30 UTC, Timon Gehr wrote:
>> On 08.04.22 17:07, Zach Tollen wrote:
>>> On Friday, 8 April 2022 at 08:23:34 UTC, Timon Gehr wrote:
>>>> Yes. I do not understand why so many people are so keen to conflate 
>>>> entirely different things into the same concepts. That's just bad 
>>>> language design.
>>>
>>> No it's not.
>>
>> Of course it is. It hampers expressiveness while adding incidental 
>> complexity. Lose-lose.
> 
> Wrong. Simplicity is fine

Your proposal is not simpler.

> when it allows you to do what you want without 
> forcing you to do what you don't want.

Your proposal forces me to do what I don't want without allowing me to 
do what I want. I don't want that.

Your proposal fails to meet your own standards.

> If what you say were true, then D 
> should have adopted every single built-in attribute and every type 
> qualifier which has ever been suggested, because the lack of them would 
> "hamper expressiveness."
> ...

I see what you did there. I can do that too: Clearly you think that 
simplicity and expressiveness are both monotone functions of the number 
of attributes that are in the language. If what you say were true, we 
should conflate @safe and pure, as well as @nogc and nothrow because 
that would be "simpler". That's ridiculous!

> The incidental complexity of a given multi-purpose feature only arises 
> when the one purpose of the feature simultaneously makes another purpose 
> of the same feature difficult to achieve. I do not consider this an 
> inevitable outcome. I want to examine the specific case instead.
> 
>>> But it's also bad language design not to be open to the possibility 
>>> of a conflation of different things which ends up being harmonious 
>>> and easy to explain.
>>
>> That's just never how it works out. You should only conflate things 
>> that are actually the same thing.
> 
> The use cases of DIP1035, for the most part, are a strict subset of the 
> uses of `__mutable`/`__metadata`.

???

`__metadata` is _much more niche_ and _much more restricted_. Almost no 
D programmer will have to use it directly.

> It is agreed that you can only access 
> `__metadata` variables from `@system` code. Therefore by implementing 
> `__metadata` you basically get DIP1035 for free.

No. I suggest you go read DIP1035 and find all the aspects not covered 
by `__metadata`.

> My proposal is simply to use DIP1035's word `@system` two encompass the two ideas.
> ...

So your proposal is to take the `@system` and `__metadata` proposal, 
concatenate them and change all occurrences of `__metadata` to 
`@system`? That would be even less sensible than what I thought you were 
arguing for, so I guess your proposal is actually not "simply" this. One 
does not simply conflate `@system` and `__metadata`.

In any case, even if your proposal was simple to state formally, it 
would still lead to really bad outcomes.

> The purpose of DIP1035 is to be able to force a variable to be 
> considered unsafe, because it is already being used in an unsafe way. If 
> you're trying to decide whether merging `@system` and `__metadata` is a 
> good idea, you would first need to come up with a use case proving a 
> possible conflict of purposes as follows:
> 
> You would need to show

You are making a proposal.

> how marking a variable `@system` because it is 
> *already* being used dangerously might then lead to it being used 
> dangerously *differently*, but now in an unintended and undesirable way, 
> solely because before, when it was used dangerously, it was immutable.
> ...

So you want to argue that everything that's potentially dangerous is 
actually the same thing? Can as well just turn off type checking and 
bounds checks in `@system` code completely at that point. It's such 
fallacious reasoning...

> It seems like a really tall order to me.

To the extent that I can make sense of your paragraph, it's very easy 
and Paul has already done it.

NB: I really dislike it when some people go from "there are no 
counterexamples" to "there is exactly one counterexample and I don't 
care about that particular one" after they have been presented a 
counterexample to their claims. It makes me not want to engage with 
their posts anymore. It's so disrespectful to other people's time. If 
you thought there were no counterexamples and were presented one, 
chances are that you also missed other things.

> And if you can't provide an 
> example like that, then on what basis can you say that these two 
> features can't be combined?
> ...

I am not interested in that particular scenario as I can provide 
multiple examples that immediately kill your proposal. However, it seems 
it's not worth the effort as you will just move the goalposts with 
informal nonsense such as "for the most part" and "in nearly all cases" 
or "used dangerously". The qualifiers have precise formal meanings.

> My main point is that mutability, in nearly all cases, is precisely what 
> would motivate a variable to require being marked `@system`.

Even in cases where mutability is the issue you should not simply be 
able to bypass `const` implicitly/by accident, even in `@system` code...

> As to global data which is initialized to unsafe values and therefore inferred 
> `@system`, I don't even know what these would be for. They seem like 
> they would always be bugs to me.

I agree with the point that it seems that you don't know existing 
language rules and use cases well enough.

Anyway, for anyone who's actually interested, here are some of the 
problems with the proposal (very likely not exhaustive, it's such an 
arbitrary and badly motivated conflation that we'd probably find new and 
horrible consequences for years to come. I have already spent more than 
two hours on this post and don't want to think about this any more, it's 
such a pointless waste of time):

- There is nothing about `@system` that screams "this is mutable", it's 
a very unintuitive language rule. You would literally state `@system 
immutable` and it would be ignored.

- `@system` variables will be a reasonably common occurrence in 
low-level code. Explicit `__mutable` is much more specialized and 
restricted in how it can be applied. It does not make sense to extend 
`@system` to encompass this much more narrow use case as most `@system` 
users won't want it.

- `@system` can be inferred from an initializer. It is then _very 
confusing_, if `const`/`immutable` are just stripped away from the 
variable declaration, _especially_ if it does not happen often.

- By conflating `__metadata` and `@system`, the compiler seems to engage 
in implicit, unsafe and unnecessary type-punning. `immutable` would be 
stripped away _transitively_ from an _existing value_ upon 
initialization. There is possibly another design where `@system t = 
foo()` would just not compile because you can't assign the `immutable` 
result of `foo()` to the mutable `t`, but that makes similarly little sense.

- Templated code may have logic that differs based on the mutability of 
the argument (e.g., copy the values of all mutable fields from one 
instance to another).

- Variables inside templates may not always be inferred `@system` the 
same way for different template arguments. Now suddenly you sometimes 
get completely unsafe, completely implicit type-punning, potentially you 
can't even tell from the outside. If you are "lucky", your code might 
not compile because you can't assign a `T v = init();` back to a `T w` 
or something.

- Memory safety may actually depend on a `@system` field keeping its 
value, so it's useful to have `immutable @system` fields.

- There can be mutable and immutable instances of the same type, some of 
them static. If a type has `@system` fields, the proposal would prevent 
immutable instances from being stored in the read-only data segment.

- Other optimizations based on `immutable` similarly still work even 
with `@system` values. (Unless `@system` is arbitrarily conflated with 
`__metadata`.) You should not have to choose between fast code and 
soundness of `@safe` checks.