I wish all qualifiers were revisited with an eye for simplification

[Timon Gehr]

General disclaimer: My post may contain rebuttals to low-level technical 
points which were made in support of some high-level claims. Apparently 
there has been some confusion: When I disagree with a technical point, I 
am not automatically taking a stance against the corresponding 
high-level claim.

On 02.08.20 22:50, Andrei Alexandrescu wrote:
> (Background: qualifiers were introduced following my horror when I 
> started writing D1 code and saw that strings are represented as char[]. 
> So structs with string members would look like:
> 
> struct Widget
> {
>      int x, y;
>      char[] name, parentName;
>      ...
> }
> 
> I found it just shocking that following a Widget's construction whoever 
> aliased the same strings the outside could change members of the Widget 
> without Widget knowing. Of course, other D1 coders disliked that as 
> well, so they'd defensively duplicate in the constructor:
> 
> struct Widget
> {
>      int x, y;
>      char[] name, parentName;
>      this(char[] n, char[] p)
>      {
>          name = n.dup;
>          parentName = p.dup;
>      }
>      ...
> }
> 
> thus ensuring proliferation of the garbage whether duplication was 
> needed or not.
> 
> I found this absolutely maddening, to the extent I didn't think D could 
> be ever used at any considerable scale while dragging this anchor behind 
> it.
> 
> The second problem was Walter was adamant about using arrays of 
> characters at strings. He found the notion of a library-defined string 
> type (a la C++) an absolute abomination. Stubborn about it like I've 
> never seen him before or after. So my unstoppable requests for a string 
> type were met with the proverbial immovable refusal. Ironically, much of 
> his argument came from an efficiency angle, yet the unnecessary 
> duplication was way less efficient than some reference counting/small 
> string optimization/etc scheme that a dedicated string type would use.
> ...

At the cost of efficient slicing?

> Then we figured things would work out if we arranged things such that 
> people could NOT change individual characters of a string. That would 
> allow sharing without the danger of long-distance influence. After many 
> discussions with Walter, Bartosz, Eric, Brad, and myself, immutable and 
> const were born.
> ...

And it has to be said that they do a great job at preventing individual 
chars in a `char[]` from being mutated. :-)

> Then followed the other qualifiers, in order: shared and inout.)
> 
> * * *
> 
> The result is... there: https://dlang.org/spec/const3.html. It has the 
> images https://dlang.org/images/qualifier-combinations.svg and 
> https://dlang.org/images/qualifier-conversions.svg and a large table and 
> a lot of rules. Whenever I code anything generic, I find myself going 
> back to those damn images and tables way more than anyone ought to. (And 
> it's ironic... I made those. Woe to the relative newcomer.)
> ...

Well, those tables and figures are easy to make, because they are based 
on simple rules. The combinatorial explosion is artificial.

> It's all too complicated, making generic D programming into 3D chess 
> instead of the difficult endeavor it already is. And what does it buy 
> us? Well, we don't need to define a string library type. Yowzers. 
> (Actually we should if we want to get rid of the GC. But then Walter 
> would oppose that. So - stalemate once again.)
> ...

In general, Walter opposes memory ownership to be mediated by library types.

> Far as I can tell, the power/weight ratio of qualifier is very poor. I 
> wish a DIP would revisit qualifiers with a stated intent to simplify 
> them as much as possible. Whenever I code generically I invariably run 
> into these issues:
> 
> * Whatever I do, however I twitch, immutable finds the opportunity to 
> lodge itself in a soft part of my body and cause constant pain. This 
> doesn't work, that doesn't work. No solution for "tail immutable" - 
> mutable references to immutable class instances can't be done without 
> contortions. Can't assign to out immutable class references, though 
> there's no reason for that (see Adam's recent post).
> ...

Personally I don't use it where it causes those problems, but maybe the 
standard library is not afforded those conveniences.

> * No matter how I dice any significant piece of code, there will be five 
> casts from immutable and/or back that I can't rid of and lose sleep at 
> night trying to convince myself are justified.
> ...

Maybe `immutable` is overused in that code base.

> * Every time "inout" comes within a radius of a mile of what I'm doing, 
> it starts to stink like a skunk. I wish I could get a restraining order. 
> I can't instantiate "inout" variables, so writing any tests or 
> constraints becomes an advanced matter of defining functions and crap.

I have never understood the `(inout int){ T.init; }` idiom. Just use `(T 
value){ value; }`.

> I get frustrated, I protest to this forum, and immediately a cabal is 
> raised under Timon's leadership. The cabal convinces me that inout is 
> actually great and that I'm an idiot. I do get convinced, which is more 
> of a proof that Timon is very good, than a testament to the conviviality 
> of inout. Then I leave and get back to my code, and it stinks of inout 
> again. And I hate it and myself for having to deal with it.
> ...

I am sure you are sincere, but I still think this is a 
misrepresentation. I don't think I ever claimed that `inout` is great. I 
merely understand what `inout` is supposed to be, but it comes way 
short. See all of the issues I have opened that show that type checking 
for `inout` is broken. When I tried to document inout properly in 2018 I 
found multiple new type system holes, I think they are open to this day.

I'm attaching my draft write-up on `inout` from 2018.

> * Nobody - probably not even Timon - knows what "shared" does or is 
> supposed to do and not do. The most I got from Walter ever is "shared is 
> intentionally restricted so you can't do much without a cast". Yet the 
> definition of "much" and the conditions under which casting is legit are 
> not anywhere to be found.
> ...

Well, I think I know what it is supposed to do, but I think Walter does 
not fully agree, as evidenced by the `scope shared` discussion. However, 
think of it this way: it is clear what _unshared_ does; it's a type 
system assertion that there will not be concurrent accesses to the given 
memory location. Casts to and from shared have to uphold that. That's 
essentially it. Then, semantically, shared variables are just standard 
variables from C or C++. Of course, that means `shared` variables should 
not be accessed using unsynchronized reads/writes in `@safe` functions, 
but I think Manu and Walter have been working on adding necessary type 
checks.

> * And of course, "shared" gladly partakes in qualifier combinations, 
> thus spreading its stink around in a combinatorial manner.

That's true of other language features. "Any field of any type can be of 
type `int`, so `int` is spreading its stink around in a combinatorial 
manner."

If you consider type constructors to be type constructors instead of 
flags on top of a type, I think they are easier to understand and you 
don't have to handle nearly the same amount of combinatorial explosion.

> Believe it or 
> not, the type "const inout shared T" exists. Of course, nobody knows 
> what it really means or how it could be used.
> ...

I beg to differ. It's either const(shared(T)) or immutable(T), but you 
don't know which. An use case is you have a function that may return 
references to its `shared` input, which it does not modify, and you want 
to preserve `immutable` qualifiers across function calls while also 
allowing the function to be called with mutable shared data.

Unfortunately this is only obvious if you understand `inout`, but 
unfortunately, `inout` is harder to understand than type theory concepts 
that are more general than `inout`...

> A "Define All Qualifiers" DIP would be a radical improvement of the 
> state of affairs.

My attached draft has some overlap with that.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: inout.md
Type: text/markdown
Size: 24508 bytes
Desc: not available
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20200804/6fafcc3b/attachment-0001.bin>