const ref parameters and r-value references

[Jonathan M Davis via Digitalmars-d-learn]

On Fri, 02 May 2014 08:17:06 +0000
Mark Isaacson via Digitalmars-d-learn
<digitalmars-d-learn at puremagic.com> wrote:

> I'm in the process of learning/practicing D and I noticed
> something that seems peculiar coming from a C++ background:
>
> If I compile and run:
>
> void fun(const ref int x) {
>    //Stuff
> }
>
> unittest {
>    fun(5); //Error! Does not compile
> }
>
> I get the specified error in my unit test. I understand that the
> cause is that I've attempted to bind ref to an r-value, what's
> curious is that in C++, the compiler realizes that this is a
> non-issue because of 'const' and just 'makes it work'. Is there a
> rationale behind why D does not do this? Is there a way to write
> 'fun' such that it avoids copies but still pledges
> const-correctness while also allowing r-values to be passed in?

By design, in D, ref only accepts lvalues. Unlike in C++, constness has no
effect on that. IIRC, the reasons have something to do with being able to tell
whether the argument is indeed an lvalue or not as well as there being
implementation issues with the fact that const T& foo in C++ doesn't necessary
have a variable for foo to refer to. I don't think that I've ever entirely
understood the rationale behind it, but Andrei is quite adamant on the matter,
and it's been argued over quite a few times. I don't know whether D's choice
on the matter is right or not, but it's not changing at this point.
Regardless, the problem that it generates is the fact that we don't have a
construct which does what C++'s const T& does - i.e. indicate that you want
to accept both lvalues and rvalues without making a copy.

Andrei suggested auto ref to fix this problem, and Walter implemented it, but
he misunderstood what Andrei had meant, so the result was a template-only
solution. If you declare

auto foo(T)(auto ref T bar) {...}

then when you call foo with an lvalue, foo will be instantiated with bar being
a ref T, whereas if foo is called with an rvalue, it will be instintiated with
bar being a T. In either case, no copy will take place. However, it requires
that foo be templated, and it results in a combinatorial explosion of template
instantiations as more auto ref parameters are added, and the function is used
with various combinations of lvalues and rvalues.

The alternative is to declare each overload yourself:

auto foo(ref T bar) {...}
auto foo(T bar) {...}

That doesn't require the function to be templated, and it works for one, maybe
two function parameters, but you have the same combinatorial explosion of
function declarations as you had with template instantiations with auto ref -
except now you're declaring them all explicitly yourself instead of the
compiler generating them for you.

What has been suggested is that we have a way to mark a non-templated function
as accepting both lvalues and rvalues - e.g.

auto foo(NewRefThingy T bar) {...}

and what it would do is make it so that underneath the hood, foo would
actually be

auto foo(ref T bar) {...}

but instead of giving an error when you pass it an rvalue, it would do the
equivalent of

auto temp = returnsRValue();
foo(temp);

so that foo would have an rvalue. You still wouldn't get any copying
happening, you'd only have to declare one function, and you wouldn't get a
combinatorial explosion of function declarations or template instantiations.
I believe that that is essentially what Andrei originally intended for auto
ref to be.

The problem is that we don't want to introduce yet another attribute to do
this. We could reuse auto ref for it, so you'd do

auto foo(auto ref T bar) {...}

but that either means that we redefine what auto ref does with templated
functions (which would be a problem, because it's actually useful there for
other purposes, because it's the closest thing that we have to perfect
forwarding at this point), or we make it so that auto ref does something
different with normal functions than it does with template functions, which
could be confusing, and you might actually like to be able to use the
non-templated auto ref solution with templated functions. It should be
possible _some_ of the time for the compiler to determine that it can optimize
the templated version into the non-templated one (i.e. when it can determine
that the forwarding capabilities of thet templated auto ref aren't used), but
it's not clear how well that will work, or whether it's an acceptable
solution.

Walter has suggested that we just redefine ref itself to do what I just
described rather than using auto ref or defining a new attribute. However,
both Andrei and I argued with him quite a bit over that, because that makes it
so that you can't tell whether a ref argument is intended to mutate what's
being passed in, or whether it's just an optimization (and you can't just use
const in all of the situations where you don't want the mutation, because D's
const is far more restrictive than C++'s const). Others agree with us, and
there are probably some that agree with Walter, but I don't remember at this
point. And Manu was arguing for something with scope ref to solve the problem,
though I'm not sure that he had much support on that one, and I can't remember
the details at this point. It involved having scope return types and altered
the meaning of scope slightly (though it was at least theoretically in the
spirit of what scope is supposed to be). But regradless of the merits of his
suggestion, Andrei was flat-out against it, because having scope ref would
essentially be creating a new attribute, which is unacceptable at this point.

In any case, we discussed it quite a bit at dconf last year and in the
newsgroup shortly thereafter, but we never came to a decision on what to do.
We have what probably should be the solution underneat the hood, just not a
way to express it in the language yet.

What _did_ come out of dconf last year with regards to ref was a means of
making it fully @safe like it's supposed to be by adding some runtime checks
that would only need to be used under circumstances when the compiler couldn't
guarantee that ref wasn't referring to a variable on the stack. But even that
hasn't been implemented yet AFAIK - probably in part because while those of us
in the discussion at dconf agreed on it, some of the folks in the newsgroup
weren't quite so hot on the idea.

So, yes, this is something that we should have solved by now, and we have
some good ideas on the subject, but we just haven't managed to get our act
together on the matter yet.

- Jonathan M Davis