rvalues -> ref (yup... again!)

[Jonathan M Davis]

On Friday, March 30, 2018 18:03:49 Manu via Digitalmars-d wrote:
> On 30 March 2018 at 16:09, Jonathan M Davis via Digitalmars-d
> > And honestly, I don't really want to encourage the use of const. It's
> > fine if folks use it where it really works, and it's potentially even
> > quite valuable there, but it seems like too often folks try to add
> > const in places where it's just going to cause problems. It's
> > particularly bad when folks try to add const to generic code - e.g. we
> > recently had to revert a commit to std.random which worked with dynamic
> > arrays but not other ranges because of const. And there have been other
> > cases where folks have wanted to try to make stuff in Phobos
> > "const-correct", which would cause problems. So, I'm not a big fan of
> > the idea of doing anything that would make folks want to use const
> > more.
>
> It's interesting... I recognise the general backlash against const. I
> personally just make myself a `struct Mutable(T) { ... }` which I use
> when I want to violate const ;)

Just don't do anything where you cast away const and then mutate. That
violates the type system and results in undefined behavior. And depending on
what the compiler chooses to do based on const, you could get some rather
subtle and nasty bugs.

> But if it turns out that const is useless, then we really need to
> reconsider the basic design of const >_<
> Like, what's the point? You're advocating active discouragement of
> const... why is there a feature which it's accepted is a loaded-gun?
> Handing it to people creates a high probability they'll shoot
> themselves in the feet.

const makes some sense when you want to write code that accepts both mutable
and immutable arguments, and there are times where it works perfectly fine
to use const. I see no problem with using const in such situations. The
problem is that there are a _lot_ cases where const simply can't be used in
D, because you need to mutate _something_ involved, even if it's not
something that's part of the "logical" state of the object - e.g. in D, you
can't put a mutex in a type and then have it protect anything in a const
member function, because locking the mutex would require mutating it, and D
provides no backdoors like C++'s mutable. What's const is actually const.

Also, const tends to interact very badly with generic code. Whatever API
you're duck typing has to include const as part of it, or you can't use
const - e.g. ranges can't be const, because they have to be mutated to be
iterated, and because the range API does not require that properties like
front or empty be const, no generic code can assume that even those work
with const (and for many ranges, they _can_'t be const - especially when one
range wraps another). In general, it can be quite difficult to even do
something like have a tail-const range over a container. Sometimes, const
can be made to work in more complex situations, but often it can't, and when
it can't, it's often a royal pain.

For non-generic code that can clearly treat an object as fully const without
needing any backdoors, const is just fine, and it may even help prevent
bugs. But for a _lot_ of code - especially idiomatic D code that does a lot
with templates and ranges - const simply doesn't work.

So, const can be used on some level, but anyone who tries to be
"const-correct" in D like many of us usually try to do in C++ is in for a
world of frustration.

If you haven't, I'd suggest that you read this article I recently wrote on
the topic:

http://jmdavisprog.com/articles/why-const-sucks.html

And here's the newsgroup thread discussing it:

https://forum.dlang.org/thread/mailman.581.1520247475.3374.digitalmars-d-announce@puremagic.com

> > Now, if you can convince Walter and Andrei to allow const ref to accept
> > rvalues, then fine. I think that that's definitely worse than an
> > attribute specifically for that, given how limiting const is, but it
> > wouldn't screw up normal ref in the process, which is what I'm most
> > worried about here. So, I don't think that going with const would be
> > the best solution to the problem, but it's far better than making ref
> > in general accept rvalues.
>
> Useful examples using 'return ref' have been presented, which I find
> quite compelling too. That's an interesting case of non-const ref.
> Safe to say I'm not convincing anyone of anything in any way other
> than DIP form.

Yes, at this point, convincing Walter or Andrei will require writing a DIP.
A well-written proposal that clearly doesn't have the downsides that they're
so against may have a chance with them, but without something that formally
and clearly provides the arguments, I don't think that they're even going to
pay much attention at this point. The topic has been debated to death
previously, and without a DIP, it really doesn't matter what Walter and
Andrei think, since at this point, they wouldn't introduce a language change
like that without a DIP. So, they'll just do like Andrei did in response to
this thread and tell you to write a DIP rather than be willing to discuss it
further on its own. But no matter the topic, if it involves a major change
to the language, since that now requires a DIP, there's not much point in
arguing the matter with Walter or Andrei beforehand except to help iron out
the DIP. Increasingly, they don't get involved in such discussions outside
of the DIP process, because it eats up too much of their time.

> But I'd like to understand your concern better. You say it's about
> scanning an API and understanding some details from it based on seeing
> 'ref' written there... how does a function accepting an rvalue
> interact with your visibility of the API?
> Like, your criticism is with respect to understanding the API at a
> glance... I don't understand how this proposal interferes with that in
> any way?

The only thing that interfers with that is making ref with no other
qualifiers accept rvalues. If it's const or inout or immutable or something
like @rvalue, then you can look at the API and know that ref is on the
parameter so that lvalues will not be copied, and if it accepts rvalues too
(be by it copying them to invisible variables to pass as lvalues or
whatever), then that may mean a performance hit due to the lack of a move,
but you know that the function is not mutating the argument with the
intention of giving the result to the caller. If it's const, then mutation
wouldn't be possible, and if it's @rvalue, then it simply wouldn't make
sense for the function to be providing the result back to the caller,
because that would only work with lvalues. @rvalue would indicate that the
function _could_ mutate the argument (so the restrictions imposed by const
would not apply), and relying on the argument not being mutated would be
asking for bugs, but it would clearly be ref for the purpose of avoiding
copying lvalues and not because the purpose is to mutate the argument and
provide the result.

As long as naked ref does not accept rvalues, then you can look it at and
reasonably assume that the function is intended to mutate the argument and
provide the result to the caller. It also prevents accidentally passing an
rvalue and thus avoids the bugs that would come from passing an rvalue to a
function that's supposed to be mutating its arguments.

As it stands, folks do still sometimes write functions which accept
arguments by ref to avoid copying lvalues, but because of the annoyances
with passing rvalues, most folks don't do that. And if the solution were to
add @rvalue to make ref accept rvalues, then anyone who wanted to use ref
for efficiency reasons would use @rvalue on the ref, and it would make it
that much more reliable that a naked ref indicated that the function was
supposed to mutate the argument. It also wouldn't introduce the problem of
accidentally passing rvalues to ref parameters like making ref in general
accept rvalues would.

So, unless you're proposing that ref in general accept rvalues, I'm not
necessarily criticizing what you're proposing. I do have reservations about
using const instead of something like @rvalue to indicate that rvalues
should be accepted due to the restrictions on const, but I also don't care
enough about being able to pass rvalues by ref to make a big stink about
const ref and auto ref being the only way to accept rvalues without copying
lvalues. I think that it's worth pointing out that making const part of the
solution is problematic, but if you write a DIP that Walter and Andrei
accept that uses const to indicate that ref will accept rvalues, then it's
generally not going to cause problems for me, since I won't be using it
enough for the restrictions on const to matter to me. So, I think that it's
a worse solution, but a subpar solution to this problem only affects me with
regards to the fact that I want D to be a great language (and thus would
prefer not to have subpar solutions) and insofar as I run into other
people's code that uses the technique. I might end up using it occasonally,
but for the most part, the few cases where I'd care, I've been able to use
auto ref to solve the problem.

Most of that I write works just fine passing stuff by value (and if it
doesn't, I tend to make the type a reference type to solve the problem), and
I'm not likely to do much with calling extern(C++) functions outside of
auto-generated code. So, while I acknowledge the problem and have no problem
with it being solved, it simply isn't an issue that affects me much. As
such, it will be that much more annoying if the solution makes existing
parts of the language worse.

> Sadly, I don't think I'll be able to make it to DConf this year...
> which is probably a reason for rejoice of literally everybody
> attending! :P
> It would be nice to workshop it in person though.

LOL. I don't know that anyone would rejoice about it. If they really don't
want to talk to you, they probably just won't talk to you.

But it's not always easy to get away, and it's not necessarily cheap, so
it's understandable if you can't come. Fortunately, I generally haven't had
a problem with getting the time off to go. It's the cost that generally
makes the trip problematic for me, though I've managed to make it works thus
far, and this year, I'm speaking, so it won't be anywhere near as expensive
for me as the last couple of years were.

If you can't make it this year, then hopefully you'll be able to make it
next year, wherever it happens to end up being.

- Jonathan M Davis