Attribute transference from callbacks?

[Jonathan M Davis]

On Saturday, December 7, 2024 10:54:40 PM MST Manu via Digitalmars-d wrote:
> I've been trying out an API strategy to use sink functions a lot more to
> hoist the item allocation/management logic to the user and out from the
> work routines. This idea has received a lot of attention, and I heard a lot
> of people saying this was a planned direction to move in phobos and friends.
>
> void parseThings(string s, void delegate(Thing) sink)
> {
>   //...
>   sink(Thing());
>   //...
> }
>
> We have a serious problem though; a function that receives a sink function
> must transfer the attributes from the sink function to itself, and we have
> no language to do that...
> What are the leading strategies that have been discussed to date? Is there
> a general 'plan'?
> Restriction attributes like pure/nothrow/@nogc/safe, etc need some
> expression to operate in a similar way to; where the attribute-ness of
> function matches the attributes of its delegate argument(/s).
>
> Here's the stupidest idea ever: expand inout to take an argument...
>
> void parseThings(string s, void delegate(Thing)  inout(nothrow)
> inout(@nogc) sink) inout(nothrow) inout(@nogc)
> {
>   //...
>   sink(Thing());
>   //...
> }
>
> Surely people have had better ideas? But you get the point, and this is
> basically essential to make the 'sink' pattern work at all in D.
>
> No, no templates; it's not right to generate multiple copies of identical
> functions just because something it CALLS would transfer an attribute. The
> function itself is identical no matter the state of any attribute
> transference, and so this isn't a template problem; it's a pattern matching
> problem.
>

Unless, I'm missing something (which is quite possible), you have this
backwards. The attributes on the delegate have to depend on the attributes
of the function, not the other way around - though it's certainly true that
if you have to pass a delegate with exactly the same list of attributes that
that delegate parameter has, then that's going to cause problems.

Every compiled function has a fixed set of attributes. A template can be
used to generate multiple functions, each with its own set of attributes,
but the compiled result in each case is a single function with a fixed set
of attributes (which even gets compiled into the function's mangling). And
those attributes tell the compiler what the function is promising (e.g.
nothrow means that it can't throw). And as such, if that function accepts a
delegate, that delegate must follow all of the requirements that come with
the attributes on the function it's being passed to; otherwise, calling the
delegate would violate the function's attributes. For instance, if this code
were to compile

```
void foo(void delegate(int) @system d) @safe
{
    d(42);
}
```

then it would result in foo calling @system code without @trusted being
involved, so it would violate @safe. Similarly,

```
void foo(void delegate(int) d) nothrow
{
    d(42);
}
```

would result in the function violating nothrow.

If foo were templated, then multiple versions of it could be compiled, and
you could templatize foo on the type of the delegate parameter, so each of
the compiled functions' attributes would depend on the type of the delegate
passed to the function in each case. But as long as you're dealing with a
non-templated function, you only get one version, and the delegate can't
violate the function's attributes, or the attributes would be meaningless.

So, I don't see how it makes any sense to talk about inferring the
attributes on the function based on the attributes on the delegate being
passed in unless the function is templated. The attributes were decided when
the function was compiled and have to be the same regardless of what's being
passed in.

Now, what we need to be able to do is pass more restrictive delegates to
less restrictive functions. For instance, if you have

```
void foo(void delegate(int) @system d) @system
{
    d(42);
}
```

then it should be possible to pass an @safe delegate, because an @safe
delegate would not violate @system, and since the delegate is essentially
just using a pointer, the generated code should be able to work just fine
regardless of the attributes on the delegate. As you pointed out, the
generated code should be the same. It's just that for the function's
attributes to not be violated, the type system has to prevent it from
accepting any delegates with looser attributes. So, right now, you can do

```
void foo(void delegate(int) @safe d)
{
    d(42);
}
```

and that compiles just fine, because the function is @system, and calling an
@safe delegate is allowed in an @system function. However, if we flip that

```
void foo(void delegate(int) @system d) @safe
{
    d(42);
}
```

then we get a compiler error, because calling an @system delegate in an
@safe function is not allowed. And of course, we get the same situation with
nothrow, pure, etc.

```
void foo(void delegate(int) nothrow d)
{
    d(42);
}
```

compiles, whereas

```
void foo(void delegate(int) d) nothrow
{
    d(42);
}
```

does not.

So, the compiler already makes sure that the attributes on the delegate
parameter don't violate the attributes on the function if that delegate is
called (the attributes can be totally different if it's not called, since
it's the call that's the problem, but it would be pretty atypical to pass a
delegate in and then not call it).

As such, what that leaves is the question of what the compiler does when you
pass the function a delegate with attributes which are more restrictive than
the parameter's attributes. I'm pretty sure that it used to be the case that
the type of the delegate that you passed in had to match the type of the
parameter exactly (which would then mean that we'd be screwed with regards
to being able to reasonably have non-templated functions take delegates with
a variety of attributes). However, from what I can tell, whatever it may
have done in the past, the compiler now actually does the right thing. For
instance, this code compiles just fine

```
void main()
{
    void delegate(int) @safe a;
    void delegate(int) pure b;
    void delegate(int) @nogc c;
    void delegate(int) nothrow d;

    foo(a);
    foo(b);
    foo(c);
    foo(d);
}

void foo(void delegate(int) d)
{
    d(42);
}
```

and if we change the signature on foo to

```
void foo(void delegate(int) @safe d)
{
    d(42);
}
```

then only the call to foo with the @safe delegate compiles, since the others
are @system and would violate the requirement put on the delegate parameter
(and the attributes on the delegate parameter have to be at least as
restrictive - but no more - than the attributes on the function).

So, from what I can tell, the problem that you're trying solve - that is,
have a non-templated function take a delegate where the delegate passed in
can have a varying set of attributes - already works. It's just that the
type system won't let you pass in a delegate which is less restrictive,
since that would violate the attributes. And from the looks of it, it was
implemented as a general implict conversion rather than something specific
to function calls, e.g.

```
static assert(is(void delegate(int) @safe :
                 void delegate(int) @system));
```

and

```
void delegate(int) @safe a;
void delegate(int) @system b = a;
```

compile just fine - but fail to compile if you flip the attributes.

So, unless I'm missing something here (which is quite possible), this issue
has already been solved.

Now, IIRC, there are issues elsewhere in the language with regards to
attributes on delegates - e.g. opApply doesn't handle them very well - but I
don't recally the details at the moment, since I basically never use
opApply. Timon would be able to explain. So, there may be improvements which
still need to be made with regards to delegates, but I don't think that we
need anything like what you're suggesting.

- Jonathan M Davis