Feedback Thread: DIP 1033--Implicit Conversion of Expressions to Delegates--Final Review

[Q. Schroll]

TL;DR: Fix attributes for functions taking delegates as 
parameters first. Otherwise, this DIP is useless with respect to 
the direction D is intended to go: using more attributes 
(default, inferred or explicit ones).

The DIP says: "Although this DIP renders lazy redundant and 
unnecessary, it does not propose actually removing lazy. That 
will be deferred for a future DIP." That indicates that replacing 
`lazy T` by T delegate() is fine in any case. It's not! When 
attributes are present, behavior cannot even easily be replicated 
with delegates.

Those two functions should be practically identical, right?

     T foo(T)(lazy T value) /*pure*/ { return value(); }
     T goo(T)(T delegate() value) /*pure?*/ { return value(); }

If T isn't something really funny that might be impure when 
moved, foo will be inferred `pure` always. Noticed the question 
mark I put behind goo's pure? Well, goo will be inferred `pure` 
never because the compiler sees its body calling the impure 
delegate.
So from the point of attributes, `lazy` and delegates are vastly 
different. Attribute inference is near useless for templates that 
take delegate parameters. Attributes have a hard time being 
applied to regular functions taking delegate parameters.
The programmer is stuck between a rock and a hard place:
A) Put the desired attribute(s) on the delegate
B) Overload on pure and impure delegates.
C) Ignore attributes (which is what most would do, accidentally).

Issues:
A) Makes the function not available for closures that aren't 
complying to the attribute.
B) For if the function can be inferred pure, @safe, nothrow 
and/or @nogc, make up to 16 overloads, all with the same body. No 
one wants that, not even with mixins.
C) But then: why have attributes in the first place

I saw myself giving up D because I thought attributes are one of 
the best things D has to offer, but failing it so badly is sad. 
Completely idiomatic @safe code must drop @safe because **one** 
library function just works best using a delegate (or function 
pointer).

The DIP introduces a questionable implicit conversion of T to T 
delegate() and questionable distinctions between function 
pointers and delegates on usage, but misses the point of 
attributes completely. Solving the attribute issue could make up 
the drawbacks of the implicit conversion.

That's for the problem.

---

But I have an idea for a solution.

Assuming a delegate parameter is called in the body of the 
function it's the parameter of, which is---in my opinion---a very 
reasonable assumption, it should be the caller who's deciding 
what behavior of a function is fine and which isn't. `lazy` works 
like that.

     S pureFactory() pure;
     S impureFactory();

     void goo(S delegate() value) pure { return value(); }

should compile. Even without the body visible. Here's the reason:

     void pureContext() pure
     {
         goo(&pureFactory); // compiles   (1)
         goo(&impureFactory); // error    (2)
     }

(1) Plugging in a pure delegate in an otherwise pure function 
will result in a pure operation, so the context (= the attributes 
of the caller) is satisfied.
(2) Plugging in an impure delegate in an otherwise pure function 
will result in a possibly impure operation, so the context is not 
satisfied.

     void impureContext()
     {
         goo(&pureFactory); // compiles   (3)
         goo(&impureFactory); // compiles (4)
     }

(3) See (1). Pure operations are allowed in an impure context.
(4) Plugging in an impure delegate in an otherwise pure function 
will result in a possibly impure operation, but the context does 
not care.

Type checking goo isn't complicated. The compiler just needs to 
pretend that the delegate parameter type is `S delegate() pure` 
(i.e. the original `S delegate()` with goo's `pure` attribute). 
If that is fine, goo is pure relative to its parameter.

The caller can decide on the callee's signature, the parameter's 
(delegate) type, and its own attributes whether a call will be 
valid. This approach does not need complicated analysis by the 
compiler. The question "Am I plugging in a pure delegate argument 
to a (relatively) pure (i.e. pure annotated) function?" is 
answered by signatures and types alone.

It's similar to weakly pure functions. Allowing weakly pure 
functions made more functions strictly pure. The same way, 
relatively pure functions make more functions non-relatively pure.

I have not thought in depth about nested delegate parameter 
types, i.e. functions like

     void f(int delegate(int delegate()) higherOrderParam) pure { 
... }

but solving it for the first layer will solve most applications 
of delegates. It seems easy, but no one so far has affirmed me 
the following reasoning is correct in general:

     int g() pure;
     int h();

In the body of f,

     higherOrderParam(&g);   // compiles
     higherOrderParam(&h);   // error

since higherOrderParam might call its parameter, but f knows what 
it's plugging in. So I assume that delegate types need never be 
annotated pure/whatever since the context will always be able to 
discern if the thing it's concerned about will be pure/whatever 
or not.

The only drawbacks of my solution that I know of are
* pure/whatever functions called with impure delegates that 
aren't called become assumed impure (breaking change in theory)
* pure/whatever functions aren't guaranteed to act pure/whatever 
unless all argumetns to delegate parameters are pure/whatever. 
This is a non-issue for all functions without delegate parameters 
and the correct thing for almost all functions with delegate 
parameters.
* Template functions could be surprisingly inferred pure/whatever 
when template parameters are delegate or function pointer types. 
Surprising here means to the programmer; almost guaranteed to be 
a pleasant surprise.

The breaking of the change can be seen to be rather weak since 
plugging in an impure delegate in a pure function will still be 
in a greater context where after all, that delegate will be 
called. Delegates are to be called after all.

With that change, attributes on delegate parameter mean: This 
function cannot work properly if that delegate isn't 
pure/whatever.