Feedback Thread: DIP 1041--Attributes for Higher-Order Functions--Community Review Round 1

[Q. Schroll]

On Monday, 12 April 2021 at 11:05:14 UTC, Timon Gehr wrote:
> On 12.04.21 11:38, Mike Parker wrote:
>> https://github.com/dlang/DIPs/blob/11fcb0f79ce7ec209fb2a302d1371722d0c8ad82/DIPs/DIP1041.md ...
>
> Unfortunately, it is not written too well: The reader gets 
> flooded with details way before being told what the problem 
> actually is or how the proposal addresses it.

Doesn't the Abstract explain what the problem is and give a 
general idea how it is addressed?

> As far as I can tell, this is trying to introduce attribute 
> polymorphism without actually adding polymorphism, much like 
> `inout` attempted and ultimately failed to do. I am very 
> skeptical. It's taking a simple problem with a simple solution 
> and addressing it using an overengineered non-orthogonal mess 
> in the hopes of not having to add additional syntax.

You're mistaken. You can take a look at the Alternatives for 
seemingly simple solutions. There ain't any. Because D isn't an 
immutable-all-the-way-down language like e.g. Haskell, none of 
the easy solutions are sound. You always have the problem of 
assigning the parameter in the functional unless it's `const` or 
another flavor of non-mutable.
If you don't go the `const` route, you have to deal with 
assignments to the parameter before it's called. You have to 
disallow assignments that, looking at the types, are a 1-to-1 
assignment. IMO, going via `const` is far more intuitive.

In fact, "not having to add additional syntax" was never the 
motivation for the proposal. Not having to introduce attributes 
_specific_ to higher-order function was.

> To add insult to injury, the first example that's shown in the 
> DIP as motivation abuses an existing type system hole.

I disagree that it is a hole in the type system. When having 
`qual₁(R delegate(Ps) qual₂)` where `qual₁` and `qual₂` are type 
qualifiers (`const`, `immutable`, etc.) it is practically most 
useful if `qual₁` only applies to the function pointer and (the 
outermost layer of) the context pointer while `qual₂` refers to 
the property of the context itself.
Since the language gives no non-UB way to assign the function 
pointer and the context pointer separately, it is not unsound.
[Here](https://forum.dlang.org/post/gauloixsonnnlswhbiqe@forum.dlang.org) is the space for discussing this issue. Unfortunately, it's mostly us two that care.

> `toString` is `const`, `sink` is `const`, the only reference to 
> result accessible to `toString` is in the context of `sink`, 
> but somehow `result` is mutated anyway.

See the paragraph above.

> Unsoundness should be fixed, not embraced!

Yes, I guess no one disagrees on that one, but on the question of 
it being an instance of it.

> Finally, there's this concern: The DIP assumes that the only 
> reasonable way to manipulate delegates in higher-order 
> functions involves calling them, but this is not accurate.

It assumes that the the most common use-case of non-mutable 
delegate parameters is only calling them. Returning them is 
another, but a rarer one. The DIP details that in this case, the 
author of the `compose` function needs to remember not to make 
the parameters mutable.

> ```D
> auto compose(A,B,C)(C delegate(B) f, B delegate(A) g)pure{
>     return a=>f(g(a));
> }
> ```
>
> With the proposed changes, composing impure functions suddenly 
> becomes an impure operation as soon as you abstract it into a 
> higher-order function. This is pure nonsense. If you have a 
> `pure` expression and abstract it into a `pure` function, it 
> should not become less `pure` in the process!

You did it correctly in the sense of the DIP. `compose` takes `f` 
and `g` as mutable. None of the proposed changes apply to mutable 
delegate parameters. By the changes proposed by this DIP, 
`compose` is `pure`. However, all delegates you pass to it lose 
information of attributes because you _could_ assign `f` or `g` 
in `compose`, no problem.

But as you don't intend to mutate `f` or `g` in it, you could get 
the idea of making them `const` like this:
```D
C delegate(A) compose(A, B, C)(const C delegate(B) f, const B 
delegate(A) g) pure
{
     return a => f(g(a));
}
```
Then, by the proposed changes, only `pure` arguments lead to a 
`pure` call expression.
However, `compose` is a good example why this is not an issue: It 
is already a template. Why not go the full route and make the 
`delegate` part of the template type arguments like this:
```D
auto compose(F : C delegate(B), G : B delegate(A), A, B, C)(F f, 
G g) pure
{
     return delegate C(A arg) => f(g(arg));
}
```
Unfortunately (see 
[here](https://issues.dlang.org/show_bug.cgi?id=21823)), when 
calling `compose` with a `const` declared delegate value, D's 
IFTI infers `const` for the parameter type although the parameter 
is copied. I didn't realize that when writing the DIP. This is a 
problem, but it can be fixed (probably should). To be honest, it 
was my C++ background that tricked me into thinking copying stuff 
removes `const` in any case.

It's not without solution, however, the solution being a cast or 
explicit template instantiation, none of which is nice:
```D
const dgAtoB = ...;
const dgBtoC = ...;
auto dgAtoC1 = compose!(C delegate(B), B delegate(A))(dgBtoC, 
dgAtoB); // no IFTI, no cast
auto dgAtoC2 = compose(cast() dgBtoC, cast() dgAtoB); // IFTI 
with cast
```
The cast is `@safe`.

> E.g., consider this:
>
> ```D
> auto compose2(A,B,C)(C delegate(B) f, B delegate(A) g)pure{
>     return compose(f,g);
> }
> ```
>
> With the changes proposed in the DIP, this does not even 
> compile, breaking abstraction/perfect forwarding.

No, because with mutable parameters, nothing changes. The DIP is 
_very_ clear about that.