Struggling to implement parallel foreach...

[Timon Gehr]

On 18.06.19 14:00, Nicholas Wilson wrote:
> On Tuesday, 18 June 2019 at 00:17:06 UTC, Timon Gehr wrote:
>> Unfortunately I have another paper deadline in a few weeks, but maybe 
>> I can set aside a few weekends this summer to try to fix function 
>> qualifiers.
> 
> I intend to beat you to it :)
> ...

Great. :)

> Suleyman Sahmi has located where things go right for the struct case[1], 
> so it shouldn't be too hard to manufacture a fix for the closure case.
> 
> [1]: 
> https://github.com/dlang/dmd/blob/f455995f234a2091c4482bff222a2977dbfa186b/src/dmd/expressionsem.d#L885-L886 
> 

I believe it might be easier to locate where things go right for 
`shared` and `immutable` capturing and fix the logic there. I provide 
tests below.

Note that a few missing checks and const-promotions when capturing are 
not the only issue with nested function qualifiers. If the following 
isses are fixed, we should be in pretty good shape (but probably I 
forgot about something):

0. Qualified capturing is not implemented fully correctly, it is 
particularly bad for `const`. Full test case:

void fun(inout(int)*){
     int* x;
     const(int*) cx;
     immutable(int*) ix;
     shared(int*) sx;
     shared(const(int*)) scx;
     inout(int*) wx;
     shared(inout(int*)) swx;
     const(inout(int*)) cwx;
     shared(const(inout(int*))) scwx;
     void foo(){
         int* x=x;
         const(int)* cx=cx; // ok
         immutable(int)* ix=ix; // ok
         shared(int)* sx=sx; // ok
         shared(const(int*)) scx=scx; // ok
         inout(int)* wx=wx; // ok
         shared(inout(int))* swx=swx; // ok
         const(inout(int))* cwx=cwx; // ok
         shared(const(inout(int)))* scwx=scwx; // ok
     }
     void fooc()const{
         int* x=x; // currently ok, shouldn't compile
         const(int)* x2=x; // ok
         const(int)* cx=cx; // ok
         immutable(int)* ix=ix; // ok
         shared(int)* sx=sx; // currently ok, shouldn't compile
         const(shared(int))* sx2=sx; // ok
         shared(const(int*)) scx=scx; // ok
         inout(int)* wx=wx; // currently ok, shouldn't compile
         const(inout(int))* wx2=wx; // ok
         shared(inout(int))* swx=swx; // currently ok, shouldn't compile
         shared(const(inout(int)))* swx2=swx; // ok
         const(inout(int))* cwx=cwx; // ok
         shared(const(inout(int)))* scwx=scwx; // ok
     }
     void fooi()immutable{
         //int* x=x; // error, correct
         //const(int)* cx=cx; // error, correct
         immutable(int)* ix=ix; // ok
         //shared(int)* sx=sx; // error, correct
         //shared(const(int*)) scx=scx; // error, correct
         //inout(int)* wx=wx; // error, correct
         //shared(inout(int))* swx=swx; // error, correct
         //const(inout(int))* cwx=cwx; // error, correct
         //shared(const(inout(int)))* scwx=scwx; // error, correct
     }
     void foos()shared{
         //int* x=x; // error, correct
         //const(int)* cx=cx; // error, correct
         immutable(int)* ix=ix; // ok
         shared(int)* sx=sx; // ok
         shared(const(int*)) scx=scx; // ok
         //inout(int)* wx=wx; // error, correct
         //shared(inout(int))* swx=swx; // currently error, should work
         //const(inout(int))* cwx=cwx; // error, correct
         //shared(const(inout(int)))* scwx=scwx; // currently error, 
should work
     }
     void foosc()shared const{
         //int* x=x; // error, correct
         //const(int)* cx=cx; // error, correct
         immutable(int)* ix=ix; // ok
         //shared(int)* sx=sx; // error, correct
         //const(shared(int))* sx2=sx; // currently error, should work
         shared(const(int*)) scx=scx; // ok
         //inout(int)* wx=wx; // error, correct
         //const(inout(int))* wx2=wx; // currently error, should work
         //shared(inout(int))* swx=swx; // error, correct
         //const(shared(inout(int)))* swx2=swx; // currently error, 
should work
         //const(inout(int))* cwx=cwx; // error, correct
         //shared(const(inout(int)))* scwx=scwx; // currently error, 
should work
     }
     void foow()inout{
         int* x=x; // currently ok, shouldn't compile
         immutable(int)* ix=ix; // ok
         shared(int)* sx=sx; // currently ok, shouldn't compile
         inout(int)* wx=wx; // ok
         shared(inout(int))* swx=swx; // ok
         const(inout(int))* cwx=cwx; // ok
         shared(const(inout(int)))* scwx=scwx; // ok
     }
     void foosw()shared inout{
         //int* x=x; // error, correct
         immutable(int)* ix=ix; // ok
         //shared(int)* sx=sx; // error, correct
         //inout(int)* wx=wx; // error, correct
         shared(inout(int))* swx=swx; // ok
         //const(inout(int))* cwx=cwx; // error, correct
         shared(const(inout(int)))* scwx=scwx; // ok
     }
     void fooscw()shared const inout{
         //int* x=x; // error, correct
         immutable(int)* ix=ix; // ok
         //shared(int)* sx=sx; // error, correct
         //inout(int)* wx=wx; // error, correct
         //shared(inout(int))* swx=swx; // error, correct
         //const(shared(inout(int)))* swx2=swx; // currently error, 
should compile
         //const(inout(int))* cwx=cwx; // error, correct
         shared(const(inout(int)))* scwx=scwx; // ok
     }
}



1. I can break the type system like this:

void main(){
     int* x=new int;
     struct S{
         int* delegate()pure dg1;
         int* dg2()pure immutable{
             return dg1(); // this shouldn't compile
         }
     }
     // you may think the next line is the problem, but this is actually ok:
     auto s=immutable(S)(()=>x);
     immutable(int*) y=s.dg2();
     assert(x is y); // mutable/immutable aliasing
}

The problem is that it is possible to call a immutable(T 
delegate(Args)). This is wrong. It should only be possible to call a 
qualified delegate if the delegate context has the respective (or a 
stronger) qualifier.

So

const(int* delegate()) dg = ...;
dg(); // should be error

const(int* delegate()immutable) dg = ...;
dg(); // this is fine

Note that the obvious idea of saying that the qualifier of the delegate 
transitively applies to the opaque delegate context does not work, 
because e.g. the delegate context cannot implicitly pick up a `const`, 
as the function pointer will access the context in a way that is typed 
mutable.


2. nested functions of `pure` functions are forced to be `pure`, but 
this is nonsense, they should just infer purity, like it is done for @safe:

int foo(){ return 2; }

void main()pure{
     enum x=(()=>foo())(); // Error: `pure` delegate `tt.main.__lambda1` 
cannot call impure function `tt.foo`
}
(I picked this example because here it is obvious that the error is 
nonsense, but this is a more general problem and not restricted to CTFE.)

In contrast, it works perfectly fine for @safe:

int foo(){ return 2; }

void main()@safe{
     enum x=(()=>foo())(); // ok!
}

I think it is terrible that the implementations of `@safe` inference and 
`pure` inference were allowed to diverge at all.


3. The same problem exists for @nogc, but the fix is easy. Here it is:
https://github.com/dlang/dmd/pull/9922#issuecomment-499544100

(I will create a pull request eventually, unless someone beats me to it. 
I would still need to write some tests.)


4. nested functions should infer `const`, `immutable`, `shared` and 
`inout` qualifiers based on the qualifiers of variables that they capture.


5. it should be possible to drop delegate context qualifiers:

int delegate()const dgc;
int delegate() dgc2=dgc; // this is correctly accepted
int delegate()immutable dgi;
int delegate() dgi2=dgi; // this is rejected incorrectly
int delegate() dgi3=()=>dgi(); // ugly workaround
int delegate()shared dgs;
int delegate() dgs2=dgs; // this is rejected incorrectly
int delegate() dgs3=()=>dgs(); // ugly workaround


6. `inout` should be implemented in a way that prevents scope confusion 
by design.