Rebooting the __metada/__mutable discussion

[Timon Gehr]

On 10.04.22 02:04, Zach Tollen wrote:
> On Saturday, 9 April 2022 at 17:22:38 UTC, Timon Gehr wrote:
>> Your proposal is not simpler.
> 
> It is simpler.

If `@system` and `__metadata` are separate annotations, you get:

- `@system` rules
- `__metadata` rules

If you conflate them, you get:

- `@system` rules
- `__metadata` rules
- Weird interactions between the rules
- Additional arbitrary rules to be able to pretend the rules for the 
different annotations are compatible anyway

The additional rules and weird interactions are what's causing 
complexity, in addition to the misnamed qualifier and the significantly 
enhanced area of contact between D users and inherent `__metadata` 
weirdness.

> The only question is whether that simplicity comes at too 
> high a cost.
> ...

If "simplicity" is causing costs, you are probably doing it wrong. Don't 
try to make things "simpler" than they are. It's a source of complexity 
and further ad-hoc patches.

>> Your proposal fails to meet your own standards.
> 
> I don't know if it meets my standards or not. I would need an example 
> where a variable which was either explicitly marked, or inferred 
> `@system`, and implemented as desired with that label, then changed its 
> behavior as a result of also implicitly having the privileges of 
> `__mutable`.
> 

It's not a privilege if it's undesirable. Also, you have been given 
multiple examples by now.

(You removed relevant context here.)
>> I see what you did there. I can do that too: Clearly you think that 
>> simplicity and expressiveness are both monotone functions of the 
>> number of attributes that are in the language. If what you say were 
>> true, we should conflate @safe and pure, as well as @nogc and nothrow 
>> because that would be "simpler". That's ridiculous!
> 
> No I'm saying that simplicity and expressiveness are tradeoffs.

I was objecting to your setting up of a straw man that you then 
proceeded to attack. I then simply demonstrated my ability to do the 
same, expecting it might sway you as you seem to enjoy that kind of 
reasoning yourself. Unfortunately this has not worked out.

In any case, there are very simple systems that are very expressive. You 
want to be in that quadrant.

> The Go 
> language, for example, traded a lot of expressiveness for simplicity. 
> That doesn't make it bad language design.

I don't think this is entirely the case. They made some miscalculations 
early on and are now left with a language that is neither as simple nor 
as expressive as it should have been, nor as orthogonal as they claim.

> It means they weighed the pros and the cons, and they made a decision.
> ...

It's probably one level further away from that. They didn't spend a lot 
of effort to make a good language. They just spent vastly more time 
writing the compiler and libraries and my understanding is they did a 
pretty decent job there. Also, perhaps they actively wanted to avoid 
having a cerebral community that cares about abstract things such as PL 
theory. (There was a year-long active hostility towards potentially 
simple things such as generics in the Go community. You can draw your 
own conclusions.)

>>> The use cases of DIP1035, for the most part, are a strict subset of 
>>> the uses of `__mutable`/`__metadata`.
>>
>> ???
> 
> All `__metadata` variables qualify as DIP1035 `@system`. If you added 
> `@system` to any instance of `__metadata`, it would have no effect.

Granted. Now you have shown "var is `__metadata`" => "var is `@system`", 
but that's not some sort of magical relationship between the qualifiers. 
There are multiple reasons why variables might by `@system` as it just 
collects every source of unsafety. `__metadata` is a new such source 
that we are considering adding to the language.

However, your claim above was the exact opposite: "var is @system" => 
"var is __metadata". As you said it is even a _strict subset_, you 
additionally claim that there are use cases where variables should be 
`__metadata` but not `@system`.

You seem to be contradicting yourself. It's either bad logic or bad 
communication.

> Therefore, if only `__metadata` were implemented, users could utilize it 
> to achieve the intended goals of DIP1035.

No, because DIP1035 does not alter variable types in any way. You could 
similarly say: "Just initialize your variables with unsafe values and 
get rid of the need for a `@system` annotation." This is however not a 
simplification. The compiler still needs to track all that and making it 
part of the notation makes it simpler to understand.

> (Just allow them to mark it 
> wherever they wanted instead of confining it to aggregate members, even 
> though most of the use cases are in fact aggregate members.)

This is not a complete specification...

> The question is simply whether additionally being able to violate the type 
> system would cause intractable problems.

No, not at all. Given that you apparently dismissed all the evidence in 
this thread as "tractable problems", your standard is way off in my 
opinion. E.g., I would similarly not run into _intractable_ problems if 
I chopped one of my arms off. That does not make it any less stupid even 
though one arm is clearly simpler than two arms. It would however not 
simplify my life.

You are basically stating that you will not move away from your 
position, no matter what. Digging in like that is not healthy, and this 
proposal is not actually worth spending time on, neither yours nor mine.

> I don't know. I suspect that 
> almost all use cases of `@system` are mutable to begin with. And if they 
> weren't, I would have to look the workarounds, to see if issues of 
> having undesired `__metadata` functionality caused more pain than the 
> simplicity of using only one keyword merited.
> ...

You keep claiming it's simpler. It's not. You are chasing after a prize 
that's not actually worth anything because you are using some useless, 
tunnel-vision notion of simplicity that only considers one (very 
unimportant) aspect of what potentially makes a language simple.

I would not chop off my arm even if someone offered to buy my house for 
one dollar in exchange, even though one dollar is a very simple amount. 
Avoiding that kind of "trade-off" should not be controversial.

>> So your proposal is...
> 
> My proposal is to take everything DIP1035 marks or infers as `@system` 
> and extend `__metadata` functionality to it.

I see, so if a module-scope variable is inferred `@system`, you will get 
an error because `__metadata` has been applied to a non-member variable. 
(I get that you state at other places you don't want it to work that 
way, but given that you seem to be unable to simply state what your 
proposal is, it's unlikely to actually be all that simple.)

> This may sound dangerous. 

Primarily it sounds utterly pointless. All the reasoning about why it's 
"dangerous" is only required because you refuse to accept that it's 
pointless.

> The logic is simply that this is `@system` code, and `@system` code is 
> dangerous.

Given your own standards of evidence, you will have to do much better 
than that. Please tell me how having separate `@system` and `__metadata` 
will cause intractable pain, suffering and ultimately end civilization 
as we know it. If you can't do that in a way that satisfies me, I will 
dismiss everything you say.

> But it is also powerful. That's the overarching idea. Both 
> dangerous and powerful. (D should also be working towards `@safe` by 
> default, of course.)
> 
> Again, the only question is whether having `__metadata` functionality 
> will cause intractable pain to the use cases of `@system` which don't 
> require it.
> ...

I really don't understand why you are willing to suffer everything short 
of "intractable pain" for this issue.

>> In any case, even if your proposal was simple to state formally, it 
>> would still lead to really bad outcomes.
> 
> The difference between us is that you are sure of that, and I am not.
> ...

I wouldn't say that's the whole difference.

> ...
> 
> The solution to our debate — insofar as my claims deserve any merit at 
> all — is to implement both DIP1035 and the `__metadata` DIP, with the 
> additional requirement that all uses of `__metadata` be marked `@system` 
> as well.

Not much point in that, it's just more line noise.

> Semantically, it would change nothing, and it would actually be 
> easier to understand for newcomers.
> ...

Newcomers will not have to learn about `__metadata` or `@system` 
variables for a pretty long time. Then they may come across `@system` 
variables at some point, and finally, some further minority of D 
programmers will actually need to significantly concern themselves with 
`__metadata` rules.

If you keep the qualifiers separate, the meaning of `@system` is simple 
and intuitive and nobody who would not otherwise need to will have to 
concern themselves with `__metadata`. That's a much better proposition 
for newcomers than arbitrary conflation.

> Then, five years later, to test my theory, add a compiler switch 
> `preview=deprecate__metadata`, put the `__metadata` functionality into 
> the `@system`, and make `__metadata` a no-op.
> 
> If nothing breaks, I was right. If something breaks, and the workarounds 
> aren't seamless, then you were right.
> 
...

In other words, you will just move the goalposts until you can claim you 
were right. Maybe you should care less about having been right in the 
past and more about being right in the future. Note that in my very 
first debate on this forum, I admitted that I had been wrong and that 
Andrei was right. For me, this is not about being right, it's about not 
having to go through the hassle of switching programming languages. I 
don't enjoy pointless debate, but weird language rules make it harder to 
justify why I am even using D. I need it to look like things are getting 
better (which generally they have been over the past 10 years), not worse.