Release D 2.073.0

Sat Jan 28 14:31:23 PST 2017

On 1/28/2017 6:56 AM, Olivier FAURE wrote:
> For what it's worth, here are my problems with 'return scope':
>
> - As far as I can tell, it's not properly documented. The github page for
> DIP-1000 is apparently pending a rewrite, and I can't find a formal definition
> of 'return scope' anywhere (the D reference 'Functions' page only mentions
> 'return ref', and in passing).

Yes, the documentation could be much better, and will be. Like any battle plan 
doesn't survive the first day of warfare, actually implementing -dip1000 has 
forced several adjustments. I expect more adjustments will be necessary. One of 
the larger difficulties is getting code with -dip1000 and without -dip1000 to 
play together. The bug Dicebot mentioned above is one of those. But this is the 
sort of problem any such scheme will have to deal with.

> - I personally don't like using the return keyword as anything but an
> instruction; when I'm reading code, I can have a good feeling of the code's flow
> by just looking at the indentation, the if/while/for blocks, and the
> break/throw/return instructions. I'll be the first to admit it's kind of minor
> though.

There's got to be some added syntax somewhere. I'd love to do it as pure 
compiler magic. Amazingly, if the code is all templates, that seems to work (the 
compiler can infer 'return' and 'scope' as necessary there).

> - It's an obvious monkey patch,

All new features of D have to be worked in to existing features with minimal 
disruption. We're stuck with that reality.

> and it will clearly have to be replaced at some point.
> It only addresses cases where a reference might be escaped through a
> single return value; it doesn't address escaping through 'out' parameters,

Yes it does (the general case is storing a value into any data structure pointed 
to by an argument).

> or through a returned tuple.

Yes it does (it's as if the tuple elements were fields of a struct).

In general, the idea is to understand what happens with pointers and values. 
Then, to understand how X works, reduce X to its constituent pointers and 
values, and that's how X must work.

It's like in electronics you'll get nowhere trying to understand how an 
amplifier works without thoroughly understand how voltage, current, and 
resistance works. Understanding the amplifier is built up from that.

> - It fails to enable useful features like the swap function, or storing a scoped
> value in a container (well, outside of @trusted code, but that's beside the point).

There will be a need for @system code for some things, that is correct. That's 
also true of Rust, where cyclic data structures have to be marked as unsafe, and 
functions cannot access mutable global data.

Storing a non-trivial data structure in a container is done by hiding the 
pointers in it with 'private' and providing access via the appropriate member 
functions. Safely managing memory is done with ref counted objects or GC memory.

'scope return' and 'ref return' allow ref counting containers to return 
temporary references to their internal data.

> - Because it isn't an integral part of the type system, but more of an external
> addition, it has a ton of special cases and little gotcha's when you try to do
> something complex with it. (ref parameters can't be scope, can't have pointers
> on scope values, which means you can't pass any kind of scope value by
> reference, you can't have scope types as template parameters, etc)

D is a complex language. There are LOTs of ways pointers can be handed around. 
Maybe if we throw D away and start over there would be fewer special cases, but 
we have to work it into the existing semantics.

Yes, you cannot express things in D like a pointer to a scope pointer. Time will 
tell, but in real code I've seen very little need for such things - little 
enough that it can be handled with @system code.

> The two last ones feel like the most important problems to me. If all you want
> to do is variants of the identity function, and returning references to
> attributes, then return ref and return scope is everything you need (arguably).
> If you want to store containers of scoped values, swap scope values, and
> generally treat scope as a first-class citizen, then return scope and return ref
> seem like a step in the wrong direction.

Nobody has come up with a better plan. A Rust-like system would require users to 
not just add annotations, but redesign all their code and data structures. It's 
out of the question.

> The meta problem people seem to have with 'return scope' seems more of a social
> problem. Apparently a lot of people feel like you haven't treated their concerns
> seriously; part of it is that as far as I'm aware there hasn't been a proper,
> open brainstorming on how to address lifetime analysis in D.

There has been off and on for about 10 years. Little has come of it.

Then came -dip25, which addressed the return ref problem. It worked surprisingly 
well. There was some severe criticism of it last year as being unusable, but 
those turned out to be implementation bugs that were not difficult to resolve.

'return scope' is little more than extending 'return ref' to deal with values.

> My reading of the situation, which may be completely off-base, is that you took
> inspiration from Marc Schütz's proposal, and wrote something simpler, easier to
> understand and to code with, and following the model you developed when coming
> up with inout (no templates, KISS, don't use up too much language complexity
> estate on an optional feature), then entered a cycle of gradually improving it,
> eventually making DIP-1000.

A reasonable summary.

> People who don't like the direction DIP-1000 goes towards are upset because they
> feel way too much effort is going towards refining an idea they don't agree with
> in the first place. To speak bluntly, I don't think you've addressed their
> concerns at all, and I hope you do so before 'scope return' is set in stone.

It's fair if you don't agree with my rationale, but that isn't the same as not 
addressing them at all. I believe I have addressed the issues you brought up 
here. If you'd like further clarification, please ask.

>> So, do what numerous people have done numerous times already, to no great effect?
> Please don't be hostile. When you have a communication problem, being
> passive-aggressive will only makes it worse.

I didn't write that.