foreach() behavior on ranges

[Steven Schveighoffer]

On 8/24/21 2:12 PM, frame wrote:
>> You can call `popFront` if you need to after the loop, or just before 
>> the break. I have to say, the term "useless" does not even come close 
>> to describing ranges using foreach in my experience.
> 
> I disagree, because foreach() is a language construct and therefore it 
> should behave in a logic way. The methods are fine in ranges or if 
> something is done manually. But in case of foreach() it's just unexpected.

I can't agree at all. It's totally expected.

If you have a for loop:

```d
int i;
for(i = 0; i < someArr.length; ++i)
{
    if(someArr[i] == desiredValue) break;
}
```

You are saying, "compiler, please execute the `++i` when I break from 
the loop because I already processed that one". How can that be 
expected? I would *never* expect that. When I break, it means "stop the 
loop, I'm done", and then I use `i` which is where I expected it to be.

> It becomes useless for foreach() because you can't rely on them if other 
> code breaks the loop and you need to use that range, like in my case. 
> But also for ranges - there is no need for a popFront() if it is not 
> called in a logic way. Then even empty() could fetch next data if 
> needed. It only makes sense if language system code uses it in a 
> strictly order and ensures that this order is always assured.

There is no problem with the ordering. What seems to be the issue is 
that you aren't used to the way ranges work.

What's great about D is that there is a solution for you:

```d
struct EagerPopfrontRange(R)
{
    R source;
    ElementType!R front;
    bool empty;
    void popFront() {
      if(source.empty) empty = true;
      else {
         front = source.front;
         source.popFront;
      }
    }
}

auto epf(R)(R inputRange) {
    auto result = EagerPopfrontRange!R(inputRange);
    result.popFront; // eager!
    return result;
}

// usage
foreach(v; someRange.epf) { ... }
```

Now if you break from the loop, the original range is pointing at the 
element *after* the one you last were processing.

>> It's not a bug. So there is no need to "handle" it.
>>
>> The pattern of using a for(each) loop to align certain things occurs 
>> all the time in code. Imagine a loop that is looking for a certain 
>> line in a file, and breaks when the line is there. Would you really 
>> want the compiler to unhelpfully throw away that line for you?
> 
> I don't get this point. If it breaks from the loop then it changes the 
> scope anyway, so my data should be already processed or copied. What is 
> thrown away here?
Why does the loop have to contain all your code? Maybe you have code 
after the loop. Maybe the loop's purpose is to align the range based on 
some criteria (e.g. take this byLine range and prime it so it contains 
the first line of the thing I'm looking for).

> 
>>
>> And if that is what you want, put `popFront` in the loop before you 
>> exit. You can't "unpopFront" something, so this provides the most 
>> flexibility.
>>
> 
> Yes, this is the solution but not the way how it should be. If the 
> programmer uses the range methods within the foreach-loop then you would 
> expect some bug. There shouldn't be a need to manipulate the range just 
> because I break the foreach-loop.

You shouldn't need to in most circumstances. I don't think I've ever 
needed to do this. And I use foreach on ranges all the time.

Granted, I probably would use a while loop to align a range rather than 
foreach.

> 
> Java, for example just uses next() and hasNext(). You can't run into a 
> bug here because one method must move the cursor.

This gives a giant clue as to the problem -- you aren't used to this. 
Java's iterator interface is different than D's. It consumes the element 
as you fetch it, instead of acting like a pointer to a current element. 
Once it gives you the element, it's done with it.

D's ranges are closer to a C++ iterator pair (which is modeled after a 
pair of pointers).

> PHP has a rewind() method. So any foreach() would reset the range or 
> could clean up before next use of it.

I'm surprised you bring PHP as an example, as it appears their foreach 
interface works EXACTLY as D does:

```php
$arriter = new ArrayIterator(array(1, 2, 3, 4));
foreach($arriter as $val) { if ($val == 2) break; }
print($arriter->current()); // 2
```

> But D just lets your range in an inconsistent state between an iteration 
> cycle. This feels just wrong. The next foreach() would not continue with 
> popFront() but with empty() again - because it even relies on it that a 
> range should be called in a given order. As there is no rewind or 
> exit-method, this order should be maintained by foreach-exit too, 
> preparing for next use. That's it.
> 
> You don't see a bug here?
> 

I believe the bug is in your expectations. While Java-like iteration 
would be a possible API D could have chosen, it's not what D chose.

-Steve