casting away const and then mutating

[Jonathan M Davis via Digitalmars-d]

On Friday, 24 July 2015 at 14:07:46 UTC, Steven Schveighoffer 
wrote:
> On 7/23/15 11:58 PM, Jonathan M Davis wrote:
>> On Friday, 24 July 2015 at 03:02:30 UTC, Steven Schveighoffer 
>> wrote:
>>> Basically, if we say this is undefined behavior, then inout is
>>> undefined behavior.
>>
>> inout is done by the compiler. It knows that it's safe to cast 
>> the
>> return type to mutable (or immutable), because it knows that 
>> the return
>> value was either the argument that it passed in or something 
>> constructed
>> within the function and thus safe to cast. The compiler knows 
>> what's
>> going on, so it can ensure that it doesn't violate the type 
>> system and
>> is well-defined.
>
> The compiler knows everything that is going on inside a 
> function. It can see the cast and knows that it should execute 
> it, and also that the original variable is mutable and could be 
> the one being mutated. This isn't any different.

You're assuming that no separate compilation is going on, which 
in the case of a template like you get with RedBlackTree, is 
true, because the source has to be there, but in the general 
case, separate compilation could make it so that the compiler 
can't see what's going on inside the function. It relies on the 
separate compilation step having verified the inout attribute 
appropriately when the function's body was compiled, and all it 
has to go on is the inout in the signature. If you were to try 
and implement inout yourself in non-templated code, the compiler 
wouldn't necessarily be able to see any of what's going on inside 
the function when it compiles the calling code. Without inout, in 
non-templated code, even if the compiler were being very smart 
about this, it wouldn't have a clue that when you cast away const 
on the return value that it was the same one that was passed in.

>>> An example of what the compiler can "start doing more than it 
>>> does
>>> now" would be helpful. I can't see how it can do anything 
>>> based on this.
>>
>> Well, it could remove dead code. For instance, if you had
>>
>> const(Foo) bar(T t)
>> {
>>      const myFoo = getFoo(t);
>>      auto value1 = pureFunc(myFoo);
>>      auto value2 = pureFunc2(myFoo);
>>      auto value3 = pureFunc3(value1, value2);
>>      return myFoo;
>> }
>>
>> All of the lines with pureFunc* could be removed outright, 
>> because
>> they're all pure function calls, and they can't possibly have 
>> mutated
>> myFoo. I wouldn't expect a lot of dead code like that, and 
>> maybe
>> something like that would never be implemented in the 
>> compiler, but it
>> could be as long as the compiler can actually rely on const 
>> not being
>> mutated.
>
> And my interpretation of the spec doesn't change this. You can 
> still elide those calls as none of them should be casting away 
> const and mutating internally.

You seem to be arguing that as long as you know that a const 
reference refers to mutable data, it is defined behavior to cast 
away const and mutate it. And if that were true, then if you knew 
that myFoo referred to mutable data, it would be valid to cast 
away const and mutate it inside of one of the pureFunc* 
functions, because you know that it's mutable and not immutable. 
And this shows why that isn't enough. And that is the major 
objection I have with what you're arguing here. In the general 
case, even if immutable is not used in the program even once, 
casting away const and mutating is not and cannot be defined 
behavior, or const guarantees nothing - just like in C++.

The exact use case that you're looking for - essentially inout - 
works only because when you cast it back, no const reference that 
was generated by calling the function with a mutable reference 
escaped that function except via the return value, so there's no 
way for the compiler to optimize based on the const reference, 
because there isn't one anymore. And that's _way_ more restricted 
than saying that it's defined behavior to cast away const and 
mutate as long as you know that the underlying data is actually 
mutable.

So, I don't object to the behavior you're arguing for being 
well-defined. It's that you're arguing that the fact that you 
know that it's mutable underneath is enough to make it valid to 
cast away const and mutate. And that cannot be the case, 
regardless of what the spec does or doesn't say, or we have C++'s 
const - except that it's transitive. And Walter has made it 
abundantly clear that his intention is that D's const be physical 
const and provide actual guarantees. For that to be the case, 
casting away and mutating const cannot be well-defined except in 
very specific cases where we can guarantee that we're not 
violating the guarantee that const objects are not mutated via a 
const reference.

Your case works, because the const reference is gone after the 
cast, and there are no others that were created from the point 
that it temporarily became const. So, it's a very special case. 
And maybe the rule can be worded in a way that incorporates that 
nicely, whereas simply saying that it's undefined behavior to 
cast away const and mutate would not allow it. But we cannot say 
that it's defined behavior to cast away const and mutate simply 
because you know that the data is mutable, or we do not have 
physical const, and const provides no real guarantees.

It sounds to me like we need to come up with a way to word the 
rule that allows for what you're trying to do while not allowing 
the mutating of const in general (even if the data is actually 
mutable) and get it approved by Walter and in the spec. Walter 
has made it clear on several occasions that const is supposed to 
be physical const with real guarantees, and if that is not what 
the spec says (or if it does say that but not clearly), then it 
needs to be updated. And I do agree that the case that you have 
here should be well-defined, but it could be tricky to come up 
with a way to word the rule that doesn't require a lot of 
ancillary explanation about what exactly it means. The spec needs 
to be clear, not wish-washy.

In either case, I think that it's clear that we need Walter to 
say something on the matter.

> These kinds of "maybe someone someday can think of something" 
> arguments are quite unconvincing.

The point is that if you're relying on undefined behavior to do 
whatever you're doing, what the compiler is doing could change 
later. Depending on how the compiler works now or on what 
improvements will or won't be able to be made later means risking 
writing code that will not work later. So, using undefined 
behavior and relying on the compiler's current behavior is just 
asking for trouble. I don't think that I should need to come up 
with a convincing argument about "someone someday can think of 
something," because that's the whole point of undefined behavior. 
You can't rely on it, because it's not defined.

And in the case of const, the compiler is supposed to be able to 
rely on data not being mutated via a const reference (that _is_ 
in the spec), so in almost all cases, casting away const and 
mutating cannot possibly be defined, because it would make it 
impossible for the compiler to rely on data not being mutated via 
a const reference.

In any case, clearly, I need to figure out how to improve the 
spec's explanation of const so that the situation is clear and 
get that approved by Walter. The simple fact that we're arguing 
over this shows that the spec isn't clear enough.

- Jonathan M Davis