Discussion Thread: DIP 1035-- at system Variables--Community Review Round 1

[Timon Gehr]

On 17.06.20 03:12, Andrei Alexandrescu wrote:
> * The background section should kickstart the narrative, but the general 
> remarks it makes leave the reader wanting. For example: "This can only 
> work if the language in which such types are defined has the needed 
> encapsulation capabilities."

It's a bad formulation. This is not about encapsulation.

> to which the obvious question is, "what's 
> wrong with a struct that puts that thing in a private member and brokers 
> all access to it?"
> ...

It does not. The module would. Also, you could add `@safe` code to it 
that would then in turn break memory safety. If you diff looks like this:

- @system int x;
+ int x;

That is fishy.

If you diff looks like this:

+ @safe void foo(){ x=3; }

That's not supposed to be fishy.

Furthermore, your suggestion does not solve the issue with unsafe 
initializers.

> * The second example uses:
> 
> __traits(getMember, sum, "tag") = 1; // ! ruin integrity of tagged union
> 
> The obvious conclusion here is: using __traits(getMember) breaks 
> encapsulation, therefore it should not be allowed in @safe code. ...

@safe means memory safe, not encapsulated. See reflection in Java.
For variables that satisfy invariants, the compiler cannot always figure 
out whether accesses are permissible without some annotations.

> The explanation that follows fails to convince:
> 
> * "the tag having private visibility only encapsulates at the module 
> level, while @trusted is applied at the function level, so it can still 
> be violated by functions in the same module" -> the D protection system 
> is fundamentally module level.

The documentation says @trusted code has to be audited, not all code in 
modules that contain a bit of @trusted code or no @trusted but both 
@system and @safe code.

> Construing that in an objection for just 
> this one feature creates the odd precedent that all other encapsulation 
> primitives should be changed or complemented accordingly. That's well 
> outside the scope of this one feature.
> 

@safe means memory safe, not encapsulated. private variables are 
variables that have to respect some invariant. @system variables are 
variables that have unsafe values.

> 
> * "even outside the module, __traits(getMember, ) bypasses private" -> that's an obvious bug that needs to be fixed.

It's not obvious that it is a bug (it would need to be defined to be a 
bug) nor is it obvious that making bypassing of `private` unsafe 
obviates the need for @system variables.

> 
> * <<The tagged union is just one example; there are many other situations where data should only be modified by code that "knows what it's doing">> - they call that private.

So your suggestion is @safe code cannot access any private members?

@safe code is code written by someone who potentially does not know what 
they are doing.

> 
> * And indeed the example with getPtr() illustrates an obvious bug. Safe code has no business calling into @system code.

Under current language rules, it's not @safe code. That's the problem. 
Variable initializers have no safety annotations.

> It's worth noting, again, that even if this DIP is approved, that code would continue being problematic.

No, that's only true for your supposedly equivalent workaround, the DIP 
solves that problem. It's proposed change (3).

However, I think accessing `extern` variables would also need special 
consideration.

> 
> * The paragraph about bool's problems even has a reference to an existing issue.

The issue is there is undefined behavior in @safe code. Fixing it 
requires design work as there are multiple ways to fix it.

> So how does it motivate the introduction of a new feature?

The DIP introduces a couple concepts related to @system variables and 
defines them.

> 
> * The quoted Rust discussion is the best argument in the DIP.

Not really. I don't think you will find additional arguments there.

> I don't know Rust enough to figure whether they have a solution similar to our @safe/@trusted/@system troika, i.e. I'm not sure they'd have a workaround just as satisfying as ours.

The Rust workaround would be to create some sort of cell struct with a 
private member and unsafe accessors and use that.

> The DIP should discuss that to establish applicability. (Also, that Rust proposal is quite controversial.)

It's similar to some extent but not fully. Rust has unsafe functions and 
unannotated functions, the equivalent of a trusted function is any 
function that has at least one `unsafe` block in it. Additionally, 
functions that can access certain state that unsafe functions can access 
have to be treated as trusted, but the language does not make it easy to 
track this.

That's why there is a desire for unsafe fields in Rust; to eliminate 
that last category. This is also why we want it in D, as we can't trust 
@safe code.


> 
> Speaking of workarounds, aside from fixing the bugs in @safe, I think someone in need for a system variable can use a little module with:
> 
> struct SysVar(T) {
>     private T value;
>     ref T get() @system { return value; }
>     void set(ref T rhs) @system { value = rhs; }
> }
> ...

You don't need the setter and calling it with an rvalue calls postblit 
and destructor more times than necessary.

> Embellishments are immediate (ctors, allowing void init, opAssign, etc). To the extent this type can be pried open in @safe code, these are definitely bugs in the existing language.

Not all of them, though the fact that it has a default constructor 
`this(T value){ this.value = value; }` is a bug. Maybe you can make it 
work the way you envision, but what is to stop someone from coming along 
and adding some more @safe code to that module? You won't even find the 
problem by grepping for @trusted and nobody wrote any unsafe code.

> 
> 
> * "First of all, disallowing the bypassing of private in @safe code is not sufficient for ensuring struct invariants. As mentioned in the quote, sometimes invariants need to hold onto types that are not unsafe, such as int. When there are no pointer members, then private fields can still be indirectly written to using overlap in a union, void-initialization, or array casting." -> all of these are unsafe operations. What am I missing? 

Simply that those are not inherently unsafe operations.

---
union U{
     int a;
     private int b;
}

void main()@safe{
     U u;
     u.a=3; // change b
}
---
void main()@safe{
	int x=void;
}
---
struct S{
     private int x;
}
void main()@safe{
     S s;
     (cast(int[])cast(int[1])s)[]=2;
}
---

Walter has expressed a desire to keep it that way, particularly 
adamantly for `void` initialization.