Discussion Thread: DIP 1041--Attributes for Higher-Order Functions--Community Review Round 1

[Timon Gehr]

On 12.04.21 16:44, Q. Schroll wrote:
 >
 > On Monday, 12 April 2021 at 11:05:14 UTC, Timon Gehr wrote:
 >> On 12.04.21 11:38, Mike Parker wrote:
 >>> 
https://github.com/dlang/DIPs/blob/11fcb0f79ce7ec209fb2a302d1371722d0c8ad82/DIPs/DIP1041.md 
...
 >>
 >> Unfortunately, it is not written too well: The reader gets flooded 
with details way before being told what the problem actually is or how 
the proposal addresses it.
 >
 > Doesn't the Abstract explain what the problem is and give a general 
idea how it is addressed?
 > ...

It does not. It's generic fluff. It's only marginally more explicit 
than: "there are problems and to address them we should change the 
language".

 >> As far as I can tell, this is trying to introduce attribute 
polymorphism without actually adding polymorphism, much like `inout` 
attempted and ultimately failed to do. I am very skeptical. It's taking 
a simple problem with a simple solution and addressing it using an 
overengineered non-orthogonal mess in the hopes of not having to add 
additional syntax.
 >
 > You're mistaken. You can take a look at the Alternatives for 
seemingly simple solutions. There ain't any.

I know there are, and I literally state how to do it in the quoted excerpt.

 > Because D isn't an immutable-all-the-way-down language like e.g. Haskell,

Haskell has plenty of support for mutable data.

 > none of the easy solutions are sound. You always have the problem of 
assigning the parameter in the functional unless it's `const` or another 
flavor of non-mutable.

Assignments are not the problem, it's the inconsistent interpretation of 
types using incompatible, special-cased meanings.

 > If you don't go the `const` route, you have to deal with assignments 
to the parameter before it's called. You have to disallow assignments 
that, looking at the types, are a 1-to-1 assignment. IMO, going via 
`const` is far more intuitive.
 > ...

It's a bad, non-orthogonal solution building on a compiler bug.

 > In fact, "not having to add additional syntax" was never the 
motivation for the proposal. Not having to introduce attributes 
_specific_ to higher-order function was.
 > ...

It adds higher-order specific rules to existing attributes without a 
good reason, which is a lot worse.

 >> To add insult to injury, the first example that's shown in the DIP 
as motivation abuses an existing type system hole.
 >
 > I disagree that it is a hole in the type system.

You are wrong, and I am not sure how to make that point to you. (When I 
tried last time, you just claimed that some other well-documented 
intentionally-designed feature, like attribute transitivity, is actually 
a bug.)

 > When having `qual₁(R delegate(Ps) qual₂)` where `qual₁` and `qual₂` 
are type qualifiers (`const`, `immutable`, etc.) it is practically most 
useful if `qual₁` only applies to the function pointer and (the 
outermost layer of) the context pointer while `qual₂` refers to the 
property of the context itself.

That allows building a gadget to completely bypass transitivity of 
qualifiers, including `immutable` and `shared`.

It's completely unsound, e.g., it allows creating race conditions in 
`@safe` code.

You can't say qualifiers are transitive except in this one case, that 
translates to them not being transitive at all. A lot of D's type system 
design is built on qualifiers being transitive.

 > Since the language gives no non-UB way to assign the function pointer 
and the context pointer separately, it is not unsound.
 > 
[Here](https://forum.dlang.org/post/gauloixsonnnlswhbiqe@forum.dlang.org) is 
the space for discussing this issue. Unfortunately, it's mostly us two 
that care.
 >

The obfuscated DIP is unfortunately not helping with that.

In any case, this is now the place to discuss this issue as you have 
chosen to use this bug as a basis for evolving the language.

 >> `toString` is `const`, `sink` is `const`, the only reference to 
result accessible to `toString` is in the context of `sink`, but somehow 
`result` is mutated anyway.
 >
 > See the paragraph above.
 >
 >> Unsoundness should be fixed, not embraced!
 >
 > Yes, I guess no one disagrees on that one, but on the question of it 
being an instance of it.
 >
 >> Finally, there's this concern: The DIP assumes that the only 
reasonable way to manipulate delegates in higher-order functions 
involves calling them, but this is not accurate.
 >
 > It assumes that the the most common use-case of non-mutable delegate 
parameters is only calling them. Returning them is another, but a rarer 
one. The DIP details that in this case, the author of the `compose` 
function needs to remember not to make the parameters mutable.
 > ...

I guess you mean the other way around.

 >> ```D
 >> auto compose(A,B,C)(C delegate(B) f, B delegate(A) g)pure{
 >>     return a=>f(g(a));
 >> }
 >> ```
 >>
 >> With the proposed changes, composing impure functions suddenly 
becomes an impure operation as soon as you abstract it into a 
higher-order function. This is pure nonsense. If you have a `pure` 
expression and abstract it into a `pure` function, it should not become 
less `pure` in the process!
 >
 > You did it correctly in the sense of the DIP. `compose` takes `f` and 
`g` as mutable. None of the proposed changes apply to mutable delegate 
parameters.

Fair, but that's a technicality tangential to my point, and as you may 
have been able to tell, I have certain reservations about reusing 
`const` in this fashion.

 > By the changes proposed by this DIP, `compose` is `pure`. However, 
all delegates you pass to it lose information of attributes because you 
_could_ assign `f` or `g` in `compose`, no problem.
 > ...

But that's a terrible reason to not be able to annotate them `const`. 
`const` means "this won't change", it does not mean "if you compose 
this, it won't be recognized as `pure`" and there is no clear way to get 
from one to the other. It's a textbook example of a messy non-orthogonal 
design that does not make any sense upon closer inspection.

 > But as you don't intend to mutate `f` or `g` in it, you could get the 
idea of making them `const` like this:

Yes, let's assume that was my intention.

 > ```D
 > C delegate(A) compose(A, B, C)(const C delegate(B) f, const B 
delegate(A) g) pure
 > {
 >     return a => f(g(a));
 > }
 > ```
 > Then, by the proposed changes, only `pure` arguments lead to a `pure` 
call expression.

Which was my point. This is indefensible.

 > However, `compose` is a good example why this is not an issue: It is 
already a template. Why not go the full route and make the `delegate` 
part of the template type arguments like this:
 > ```D
 > auto compose(F : C delegate(B), G : B delegate(A), A, B, C)(F f, G g) 
pure
 > {
 >     return delegate C(A arg) => f(g(arg));
 > }
 > ```

The fact that there is some ugly workaround for my illustrative example 
that also defeats the point of your DIP does not eliminate the problem 
with the DIP.

 > Unfortunately (see 
[here](https://issues.dlang.org/show_bug.cgi?id=21823)), when calling 
`compose` with a `const` declared delegate value, D's IFTI infers 
`const` for the parameter type although the parameter is copied. I 
didn't realize that when writing the DIP. This is a problem, but it can 
be fixed (probably should).

(Your reinterpretation of what delegate qualifiers mean would need a DIP 
in its own right and it would hopefully be rejected.)