Struggling to implement parallel foreach...

[Manu]

On Mon., 17 Jun. 2019, 11:40 am Timon Gehr via Digitalmars-d, <
digitalmars-d at puremagic.com> wrote:

> On 17.06.19 03:17, Manu wrote:
> > On Sun, Jun 16, 2019 at 6:00 PM Kagamin via Digitalmars-d
> > <digitalmars-d at puremagic.com> wrote:
> >>
> >> On Saturday, 15 June 2019 at 06:13:09 UTC, Manu wrote:
> >>> struct S
> >>> {
> >>>    void method() shared;
> >>> }
> >>>
> >>> void test()
> >>> {
> >>>    S s;
> >>>    void localFun() shared
> >>>    {
> >>>      s.method(); // <- s must transitively receive const from the
> >>> context pointer, otherwise this doesn't work
> >>>    }
> >>> }
> >>>
> >>> Current semantics would reject this, because s is not shared,
> >>> and
> >>> therefore not accessible by localFun.
> >>> The unnecessary complexity inhibits the only useful thing that a
> >>> shared function can do.
> >>
> >> Declare `s` as shared and it will work. It would receive
> >> qualifier from context, but unshared variables can't be captured
> >> by shared closure to begin with because unshared data can't part
> >> of shared context, because that would break the guarantee that
> >> unshared data shouldn't be shared. Currently compiler implements
> >> check which variables can be captured and which can't.
> >
> > I don't think the capture should be shared, only the function argument
> > has the shared attribute applied, ie:
> > Capture cap;
> > void localFun(shared(Capture)* ctx);
> > localFun(cap); // <- error: can't pass `Context*` to
> > `shared(Context)*`... form there, we can begin to do something
> > interesting.
> >
> > In you consider the const case, this will work perfectly fine, and
> > everything will be as expected with absolutely no magic of any kind.
> > immutable local functions can not be called under any circumstances;
> > this seems like common sense to me.
>
> It's not common sense, it's nonsense. It should be common sense that it
> is nonsense. I don't understand how you can possibly reach that
> conclusion. Closures with an immutable-qualified context can only
> capture immutable variables. What is so surprising about this?
>

What's the use of that? Can you point me at any code like that?
Make a const local function if you want to stop it mutating stuff... what's
interesting about an immutable local function?

> inout is fine too, just like const.
>
> Absolutely not. You can't implicitly promote stuff to `inout`.
>

What are you taking about? inout accepts mutable, const, or immutable. I
don't know what you could mean?
Context objects are mutable, unless called via another local function where
they would have gained the respective qualifier, and that would interact
with inout correctly, eliminating some existing false-mutability bugs.

> shared is the interesting one; we shouldn't be able to pass the
> > capture to the function because Context* -> shared(Context)*, but we
> > can start to talk about ways this can work.
> > One way is that the promotion is actally perfectly valid! Because you
> > are calling a local function from the local thread; so the shared
> > function will have a shared reference to the local context only for
> > the life of the function call, and when it returns, the shared
> > reference must end with the function. We can do this by declaring the
> > local function: `void localFun(scope shared(Context)* ctx);`
> > A shared local function is valid so long as that function does NOT
> > escape any of those shared references. It might want to attribute
> > `return` too.
>
> This idea is not at all contingent on the nonsensical


How is uniformity nonsense? Are you saying that qualified methods are
nonsense? How could applying proven, uniform, and predictable rules be
considered nonsense?
Can you show me how these special-case rules are useful? Qualified local
functions are extremely rare; I suspect someone just dun some design that
sounded cool and felt chuffed, but probably never used it.

parts of your
> vision, but I don't know if there is a simple way to make this work. It
> will require more care.
>

I agree there's certainly no simple way to do it assuming the current
design; I did try and imagine a solution extensively under the existing
implementation for quite some time, but I don't believe it's possible
without effectively transforming it towards the default semantics.
...which isn't actually surprising! You shouldn't be surprised that uniform
semantics interact properly with all the other existing language. If the
design just worked like any normal qualified this pointer, there'd be
nothing to think about other than how to call the shared function, which I
think is a problem we can defeat. Everything else would work elegantly with
no special code in the compiler.

>>From there, parallel foreach can be implemented as a @trusted function.
> >
>
> So your evil master plan is:
>
> 1. Make qualified local functions useless.
>

Not at all, const and inout work as expected (surely the 99% case), and
reflective meta will actually work too. Existing bugs are all eliminated
instantly. `immutable` has no useful meaning anyway (just use const? The
thing about local functions is that they're local. It's not API material),
shared is the only interesting case that's affected, and I can't imagine
another way to make it useful.

2. Make qualified local functions more useful.
>

If you can do this without 1, then great. I couldn't imagine a solution.
The problem is specifically the non-uniformity and special case rules.

3. Profit.
>
> Just skip phase 1, please.
>

I've asked a couple of times for a demonstration of how existing semantics
are useful? It's certainly complex, non-uniform, and surprising.
The closure has un-qualified objects in it despite the function
qualification, which means typeof() doesn't work right, and derivative meta
that depends on that is broken by consequence.

Explain to me how qualifying the context makes it useless? How does it make
anything that works right now stop working?
I think that makes it uniform, only meaningfully affects shared (in a
positive way), and removes a mountain of complex special case code.

But you know what, do what you want, I'll shut up... If it just gets fixed,
I won't complain.

>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20190617/9fbf93ba/attachment-0001.html>