Closures and loop scope

[Timon Gehr]

On 06/04/2013 11:57 PM, Nick Sabalausky wrote:
> On Tue, 04 Jun 2013 22:52:27 +0200
> Timon Gehr <timon.gehr at gmx.ch> wrote:
>
>> On 06/04/2013 09:19 PM, Idan Arye wrote:
>>> Consider the following code. What will it print?
>>>
>>>       auto arr=new ulong delegate()[5];
>>>
>>>       foreach(i;0..arr.length){
>>>           arr[i]=()=>i;
>>>       }
>>>
>>>       writeln(arr.map!`a()`());
>>>
>>> It is natural to expect that it will print [0, 1, 2, 3, 4], but it
>>> actually prints [5, 5, 5, 5, 5]. The reason is obvious - all the
>>> delegates refer to the same `i` in their closures, and at the end
>>> of the loop that `i` is set to `5`.
>>>
>>> ...
>>
>> It is not that obvious. They refer to different i's that happen to
>> reside at the same place in the stack frame. It's a bug.
>>
>> It is more obvious that it is a bug given this code snippet:
>>
>> import std.stdio, std.algorithm;
>>
>> void main(){
>>       auto arr=new ulong delegate()[5];
>>       foreach(immutable i;0..arr.length){
>>            arr[i]={auto j=i;return {assert(j==i); return i;};}();
>>       }
>>       writeln(arr.map!`a()`());
>> }
>>
>> As you can see, 'i' mutates even though it is declared immutable.
>>
>
> Keep in mind that this exhibits the same "mutating immutable" behavior:
>
> // Prints 0, 1, 2, 3, 4, even though i is immutable
> foreach(immutable i; 0..5){
>      writeln(i);
>


No, it does not! My code contains immutable variables j and i (in fact, 
5 pairs of these), where with DMD's buggy behaviour it appears that at 
one point in time j==i and at another point in time j!=i.

Your code contains five distinct immutable variables i with different 
values.

> It's questionable as to whether that's really a problem. And even if it
> is a problem, it would *only* be because you make i immutable. So
> this is irrelevant to the OP's examples because:
>
> 1. He didn't use immutable, and
> ...

No, this point is irrelevant. It is a bug in any case. If immutable is 
used, this bug can be exploited to break the const system. Hence, I was 
using immutable to make my point more clear. Apparently it had the 
converse effect.

> 2. A delegate is *expected* to read the values inside its closure at
> the time of delegate *invocation*, not at the time of delegate creation.
>

Obviously. What you seem to miss is that there is no aliasing between 
any of the delegate closures in the above code.

Note that I understand exactly what you think is happening.

>>> ...
>>
>> Yes. Yes.
>>
>> http://d.puremagic.com/issues/show_bug.cgi?id=2043
>
> That refers to a local defined within the loop, not the iteration
> variable itself, which is different from the OP.

No, it is not, as I have explained before. Since 2.063, the loop 
variable that is exposed from the foreach loop behaves like a for-loop 
loop body-local variable.

> Also, it's
> questionable that there's any problem there *other* than the immutable
> stuff.
>

Obviously there is. The only reason why the code appears to behave as 
you assume it does is because the loop-local variables happen to be 
allocated at the same place for each iteration. It's a memory safety 
issue. Allocated memory is allocated again before it is freed.

If that helps, other C-like languages with closure support (eg. C#) 
correctly allocate a closure context per loop iteration.