Closure capture loop variables

[Adam D. Ruppe via Digitalmars-d]

On Friday, 1 May 2015 at 17:51:05 UTC, Walter Bright wrote:
> Yes, they are.

I thought this until just a couple weeks ago when I was shown to 
be pretty conclusively wrong. See the discussion here:

https://issues.dlang.org/show_bug.cgi?id=2043

When a new scope is introduced, a new variable is created. It 
might happen to share memory as an optimization in the 
implementation, but it is conceptually a whole new variable.

foreach(i; 0..10) {
    int a; // new variable declared, it is set to 0 right now
    assert(a == 0); // always passes
    a = 5; // this isn't kept on the next iteration through
}

When you capture a variable from an inner scope, the optimization 
of sharing memory with the same variable on a previous iteration 
is no longer valid because the old variable now continues to 
exist.

The correct behavior is analogous to:

{
   auto a = new Object();
}
{
   auto a = new Object();
}

There, the GC might collect the first a and reuse the memory for 
the second a, but they are still different a's.

When you do a closure, you're doing:

Object capturedVariable, otherCapturedVariable;

{
    auto a = new Object();
    capturedVariable = a;
}
{
    auto a = new Object();
    otherCapturedVariable = a;
}

Note that this is exactly what happens now if you call the 
function twice, but a scoped variable inside a loop is the same 
idea.

If the GC collected the first a and reused its memory for the 
second a, that'd be a bug - there's another reference to it in 
capturedVariable, so the memory is not safe to reuse.



Javascript does D's current behavior, so I thought it was correct 
too, but C# doesn't it that way. And thinking about it, 
Javascript doesn't really do it that way either because it's 
`var`s are hoisted up to function scope anyway - there's no such 
thing as a variable whose lifetime is only inside a loop there.

(Note: the new `let` keyword in javascript is supposed to do 
scoping... but has the same closure behavior as `var` in firefox. 
However, looking at the docs, this seems to be a bug (perhaps in 
my test, or perhaps in my oldish version of firefox. Take a look: 
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Closures

"Prior to the introduction of the let keyword in ECMAScript 6, a 
common problem with closures occurred when they were created 
inside a loop. "

The let keyword, which adds lexical scoping rather than hoisted 
to function scoping, is said to change this situation. D's 
variables all work like `let` in JS. Therefore, we should do what 
it does too, which is what C# also does.)

> D closures capture variables by reference.

If this is the standard, D's implementation is still wrong. It 
isn't capturing the inner variable by reference, it is capturing 
the reused memory by reference. It is analogous to the GC 
collecting and reusing memory that is still referenced in an 
outer scope - a clear bug.

The D standard says "The stack variables referenced by a nested 
function are still valid even after the function exits (this is 
different from D 1.0). ", so arguably you could say it is doing 
the right thing and capturing the stack, something I agreed with 
again until just ten days ago.

See my change of mind here too in the edit: 
http://stackoverflow.com/questions/29759419/closures-in-loops-capturing-by-reference/29760081#29760081

There, I say it is expected because a longstanding bug is 
expected to work around.... but that doesn't make it *right*.