DIP69 - Implement scope for escape proof references

[Walter Bright via Digitalmars-d]

On 12/6/2014 4:49 PM, Manu via Digitalmars-d wrote:
> On 7 December 2014 at 08:15, Walter Bright via Digitalmars-d
> There's a lot more more to it, but perhaps I'm among few who have had
> such extensive need to engage in these tasks?

I don't know, but it's a mystery to me what you're doing that there's no 
reasonable alternative, or why this is such an extensive issue for you.


>> This is incorrect, you had enormous influence over the vector type in D! And
>> I wish you had more.
>
> I appreciate that. That was uncontroversial though; I didn't need to
> spend months or years trying to justify my claims on that issue. I
> feel like that was a known item that was just somewhere slightly down
> the list, and I was able to bring it forward.
> I was never in a position where I had to argue against Andrei to sell that one.

UDA was controversial, and was one of your initiatives.


> I have std.simd sitting here, and I really want to finish it, but I
> still don't have the tools to do so.
> I need, at least, forceinline to complete it, but that one *is*
> controversial - we've talked about this for years.

I proposed a DIP to fix that, but it could not get reasonable consensus.

   http://wiki.dlang.org/DIP56

You did, after all, convince me that we need an "always inline" and a "never 
inline" method.



> There is also a practical problem with GCC (perhaps it's an
> incompatbility with my std.simd design), where I need to change the
> sse-level between functions.
> There is a GCC attribute to do this ('target'), but it would rely on,
> at least, a few subtle tweaks. In this case, I need to be able to feed
> a template argument to a UDA, which doesn't work because UDA
> declarations seem to be parsed prior to knowledge of functions
> template args. There may be a further problem with GCC though which I
> haven't been able to prove yet though.

I don't know why this needs to be a deduced template argument.


> The reason for this is that an important design goal for my work was
> to be able to semi-automatically generate multiple code paths for
> different SSE versions, which can be selected at runtime based on
> available hardware.

No argument there.


> I need a little more language support. I've reached out a few times,
> but there hasn't been too much interest in the past. I really do wanna
> finish it one of these days though!

I want it done, too :-)


>> I don't really know what you want. Well, perhaps a better statement is
>> 'why', not what.
>
> Because a function is a function. It's a fundamental and ultimately
> *simple* element of a language, with simple and reliable in behaviour;
> you write a function, compiler emits some code with a matching symbol
> name. There is perhaps nothing more simple or fundamental to the
> language.
>
> A template is not a function. It may *generate* a function, but now
> we've added many extra details which require careful consideration and
> handling, like where the code is, or isn't, emitted? How many
> permutations are there? What about static/dynamic libraries? Calling
> with various, or arbitrary types, always required careful
> consideration, more so than an explicit type.
> Templates are a powerful (often dangerous) **tool**, which should only
> be applied deliberately and very carefully.
> Obviously auto-ref gives hard answers to some of those questions, but
> it is still a template, and that instantly makes it much more complex.

I view a template function as a function that takes both compile-time and 
run-time arguments, not as a function generator. (The latter is an 
implementation artifact.)


> Making a function that has nothing to do with templates into a
> template is not what I want... it couldn't be further from what I ever
> wanted!

I don't see any problem with:

    T func()(runtime args ...) { ... }

That idiom has found various uses in practice.


> The ABI is different; there are 2 functions now (or zero, until one is
> called). Thich means taking the address of a function with auto-ref is
> a complex issue.
> ref-ness isn't a decision I ever want the compiler to make for me.

But why?

> I either want ref, or I don't, and I will state that to the compiler
> explicitly.
> This is especially true when interacting with other languages; while D
> remains in relative infancy, I think that is an overwhelmingly common
> situation when a project is modestly large.

The set of other languages that support ref parameters is [C++]. None others I 
know of that are accessible from D. Is it commonplace in C++ to have these 
overloads:

     T foo(S s);
     T foo(S& s);

? Such a practice would at least raise an eyebrow for me.


> In the situation where templates are involved, it would be nice to be
> able to make that explicit statement that some type is ref or not at
> the point of template instantiation, and the resolution should work
> according to the well-defined rules that we are all familiar with.
> Within a complex template, if logic should be performed, we have
> powerful tools to do that already; nobody ever complains about
> Unqual!, or PointerTarget!... these things are very easy to read and
> understand. ref should fit right in there with the others, is(x ==
> ref), UnRef!T, etc.

http://dlang.org/traits.html#isRef

> Instead, it's practically orthogonal to the language. Wrangling
> anything involving storage classes is really, really tedious, and
> almost always results in extensive code duplication, or text mixins
> when duplication just get too much.
>
> This issue really surprised me at the time, because the whole thing
> was in response to a topic that I consistently raised and pushed.
> Who was the customer welcoming that solution? I could never work it
> out; there was nobody else that seemed particularly invested in the
> problem, except the usual suite of language enthusiasts that took it
> as an interesting intellectual problem to discuss. The person who
> cared about that issue the most (afaict) was completely unsatisfied
> with the solution. I said at the time that I would have preferred that
> no action was taken, than to compound, and further set in stone, an
> issue that I was already extremely critical of.
>
>
> Don't double down on this mistake with scope!

The thing is, I don't understand *why* you want to wrangle storage classes. What 
is the coding pattern?


>> Why does your code need to care so much about to ref or not to ref? That's
>> the central point here, I think.
>
> That same logic could ask why I want to have 'short', or 'int' in some
> places, but not others. There's no absolute answer. As a software
> engineer, and particularly, a native software engineer, it's a
> fundamental part of the type system (sorry, not part of the type
> system!) that I must have control over.

I can give practical technical reasons why for 'short' or 'int'.


> Why offer 'ref' if you don't intend people to use it?
> I put to you, why do you hate 'ref' so much? Remove it from the
> language if people aren't meant to use it.

That's the wrong question. I could ask "why do you hate ref as a storage class"?


> ref is just a pointer with some semantics removed (pointer
> re-assignment, indexing). It's use facilitates some forms of generic
> code which would fail otherwise, and also reduces some possibilities
> for end-user misuse or mistakes when using pointers (ie, user may
> intend an assignment, but unsuspectingly re-assign a pointer; result
> looks the same, but exposes bug somewhere else).
>
> It's also used as an optimisation where I require control over the
> ref-ness of things, but don't want to retrofit the code with a sea of
> '*' and '&' operators.
>
> It's also used when interfacing C++, where it appears in API's extensively.
>
> I'm sure there are many more common and useful cases where ref is a good tool.

And D's ref works for all that.


>> The disadvantages of making it a type qualifier are:
>>
>> 1. far more complexity. Type constructors interact with everything, often in
>> unanticipated ways. We spent *years* working out issues with the 'const'
>> type qualifier, and are still doing so. Kenji just fixed another one.
>
> What you describe is is the opposite of 'complexity'. Uniform
> behaviour is what people expect from a language.
> I understand complexity may arise as a result of interaction with
> other features, but that's a worthwhile issue to explore if you ask
> me. That's complexity that can actually be addressed, rather than
> pushed to the side.
> Edge cases which are orthogonal to the rest of the language (ie, ref),
> are a terrible idea. You don't buffer against complexity by creating a
> whole new class of complexity, which we have no effective tools to
> manage or mitigate.

Uniform behavior is a nice ideal, and sounds good, but it never happens in 
practice with programming languages. We can't even get 'int' to behave 
uniformly! (Quick: what is -int.min ?)

You might also consider the "uniformity" of the ref type in C++. It's awful - 
it's a special case EVERYWHERE in the C++ type system! It just does not fit as a 
type qualifier.


>> 2. we are never going to get users to use 'scope' qualifiers pervasively.
>> It's been a long struggle to get 'const' used.
>
> I don't think scope will be as important as const. I think we should
> *try it*, to experimentally see how it plays out.
> That said, scope should be able to be inferred quite effectively in
> many most cases. Again, I think we will only be able to measure the
> effectiveness of this when we try it.

'scope' is so important we may even consider it to be the default.


>> 3. we added 'inout' as a type qualifier to avoid code duplication engendered
>> by 'const'. It hurts my brain to even think about how that might interact
>> with 'scope' qualifiers.
>
> inout was an interesting idea (with a terrible name!). I'm still not
> sure if I think it was a good idea or not, but I have found it very
> useful.
>
> What's the issue? I don't quite see how inout and scope overlap (no
> differently than const or immutable?).

Every type will have scope and nonscope variants, combinatorialy with the other 
type qualifiers.


>> Yes, I agree unequivocably, that 'scope' as a type qualifier is more
>> expressive and more powerful than as a storage class. Multiple inheritance
>> is also more expressive and more powerful than single inheritance. But many
>> times, more power perhaps isn't better than redoing the program design to
>> use something simpler and less complex.
>
> I don't think that's a reasonable comparison. Multiple inheritance is
> (I think quite well agreed) a wildly unpopular and super-complex
> disaster.
>
> Comparing scope-as-a-storage-class or scope-as-a-type-constructor is
> nothing like comparing multiple inheritence and interfaces...
> interfaces aren't orthogonal to the language for a start! And they're
> well understood, and precedented in other languages.
>
> Don't introduce red herrings which incite unrelated, yet strong
> emotional response.

Nobody has successfully introduced a type qualifier like scope.


> My argument is that scope as storage class is MORE COMPLEX in that it
> is orthogonal to the language, than scope as type constructor. I think
> users will also find it extremely unintuitive, when they realise that
> all the usual tools for interacting with types in the language are
> unavailable in this special case.
> I think scope will also prove to be more popular than ref, so, while
> you don't hear so much about how much of a disaster ref is, you'll
> start to hear all the exact same problems arise when people are trying
> to use scope, because it's a far more interesting type qualifier (and
> should probably be the default).
>
> ...I can't wait for 'auto scope'! ;)

I guess we'll see! I agree that this design is untried, and we don't really know 
how it will work out.