DIP69 - Implement scope for escape proof references

[H. S. Teoh via Digitalmars-d]

On Thu, Dec 04, 2014 at 01:10:31PM -0800, Walter Bright via Digitalmars-d wrote:
> On 12/4/2014 11:41 AM, H. S. Teoh via Digitalmars-d wrote:
> >Can we pretty please use the term "type qualifier" instead of "type
> >constructor"? ;-)
> 
> I think "type constructor" is the more pedantically correct term, but
> you'll have to ask Andrei :-)

Recently there was a bug filed by Andrei himself, and a bunch of merged
PRs, that renamed "type constructor" to "type qualifier". The general
sense I got was that we're deprecating "type constructor" in preference
for "type qualifier".


> >1) Why are scope violations only reported for @safe code?? IMO this
> >greatly limits the usefulness of this DIP.  If I mark something as
> >scope, I'd expect it should be enforced by the compiler regardless of
> >@safe annotations, otherwise what's the point??
> 
> Because there are inevitably cases where you'll need to wrap unsafe
> code with a safe interface. By screwing this down too tightly for
> @system code, you'll force people to use really ugly workarounds.

Are there any use cases for bypassing scope semantics in @system code?
How common are they? Maybe such code *should* look ugly -- to draw
attention to the fact that something un- at safe is going on.

Or perhaps we can make cast() strip away scope? I much rather require
explicit annotation for potentially unsafe operations, than to have
subtle bugs creep into the code just because I forgot to mark something
as @safe.


> >Currently, due to the incompleteness of @safe, it's difficult to use
> >@safe annotations everywhere I'd like to (e.g. I need to use some
> >un- at safe Phobos functions that really ought to be @safe, but aren't
> >due to various reasons, like compiler limitations, etc.). This
> >greatly limits the usefulness of this DIP, if scope is only enforced
> >in @safe code!
> 
> Are there bug reports on this?

There are, though probably incomplete, and there's also an ongoing
stream of Phobos PR's for marking things @safe that ought not to be
@system.  There's a *lot* of Phobos code that needs to be cleaned up in
this way before this can be workable. (Unless we abuse @trusted as a
temporary workaround -- but that's not an approach I'd like to
recommend!)


> >2) Is there a way to detect if something is marked as scope?
> 
> Using __traits(compiles,...) ought to work.

I see.


> >If not, how does this proposal actually enable ref-counted types?
> >(I'm assuming a library ref-counting type here; or are we expecting
> >further compiler enhancements for ref counting?)
> 
> The ref counting would be done by a wrapper, which implicitly converts
> to the wrappee by exposing a scope ref.

Just saw your other post on this topic, makes sense.

Still not sure how it would address the RC folks' request for compiler
support, but I suppose that's a different topic.


> >3) What does scope mean for delegate parameters? To what does the
> >scope apply, the delegate itself, its body, or its return value, or
> >...?
> >
> >	struct S {
> >		void opApply(scope void delegate(ref int) loopBody) {
> >			...
> >			// what restrictions (might) apply here w.r.t.
> >			// how loopBody can be called?
> >		}
> >	}
> 
> Hmmm, looks like a problem I didn't think of. darnit!

I think it's a fairly important issue, since making this work correctly
will also open up optimization opportunities for a good chunk of
delegate-based code.


> >And what would be the effect on the caller's side?
> >
> >	S s;
> >	foreach (i; s) {
> >		// what restrictions (might) apply here?
> >	}
> 
> i would get its scope inferred as necessary, so restrictions would
> apply as they would to a 'scope i'.

I was thinking more of what implications may hold in the loop body for
references to (local) variables outside the loop, since that presents
another potential optimization opportunity if we do it right.


> >4) Under "Expressions", how does scope interact with overloaded
> >operators? Do the same rules apply to expressions that use overloaded
> >operators as they apply to built-in operators, or do they apply as
> >though the overloaded operators were written out in function-call
> >syntax?  What happens if some overloaded operators take a mix of
> >scope and non-scope parameters?
> 
> Overloaded operators are treated like calls to the functions that back
> them.

OK. You might want to state this explicitly in the DIP, just so it isn't
left up to interpretation. :-)


> >Finally, the following isn't directly related to this DIP, since
> >scope is intended to solve this problem, but I thought I should bring
> >it up.  :-) In the section "escaping via return", 5 points are listed
> >as sufficient for detecting the "return func(t);" case. The following
> >section "scope ref" states that these 5 points are "correct" (in
> >implementing escaping reference detection). However, isn't the
> >following a loophole?
> >
> >	struct S {
> >		int x;
> >	}
> >	ref int func(ref S s) {
> >		return s.x;
> >	}
> >	ref T foo() {
> >		S s;
> >		return func(s); // escaping reference
> >	}
> >
> >Since S cannot implicitly convert to int, it would appear that this
> >circumvents escaping reference detection.
> 
> No, because under this proposal, s.x is treated as if it were just s
> as far a scope rules are concerned.
[...]

I'm not quite sure I see how this could be implemented. When the
compiler is compiling func(), say it's in a different module, there's
nothing about the function that says it's illegal to return a reference
to the member of an incoming argument. First of all, this isn't a
template function so scope inference isn't in effect; secondly, even of
it were in effect (say we write it as `func()(ref S s)` instead), it
wouldn't be able to infer the return value as scope, since it's
obviously escaping a reference to a function argument. So, unless I'm
missing something obvious, func() will be accepted by the compiler (as
it should, since we didn't mark anything as scope here).

Now when the compiler compiles foo(), it can no longer see what's inside
func(), so there's no way for it to know that the ref int being returned
actually comes from the ref S parameter. Furthermore, func()'s signature
is not annotated with scope, so how would the compiler know where to
apply scoping rules?

Unless I'm missing something, this still looks like a loophole.


T

-- 
If you compete with slaves, you become a slave. -- Norbert Wiener