foreach / mutating iterator - How to do this?

[Jonathan M Davis]

On Monday, June 25, 2018 17:29:23 Robert M. Münch via Digitalmars-d-learn 
wrote:
> I have two foreach loops where the inner should change the iterator
> (append new entries) of the outer.
>
> foreach(a, candidates) {
>   foreach(b, a) {
>       if(...) candidates ~= additionalCandidate;
>   }
> }
>
> The foreach docs state that the collection must not change during
> iteration.
>
> So, how to best handle such a situation then? Using a plain for loop?

Either that or create a separate array containing the elements you're adding
and then append that to candidates after the loop has terminated. Or if all
you're really trying to do is run an operation on a list of items, and in
the process, you get more items that you want to operate on but don't need
to keep them around afterwards, you could just wrap the operation in a
function and use recursion. e.g.

foreach(a, candidates)
{
    doStuff(a);
}

void func(T)(T a)
{
    foreach(b, a)
    {
        if(...)
            func(additionalCandidate);
    }
}

But regardless, you can't mutate something while you're iterating over it
with foreach, so you're either going to have to manually control the
iteration yourself so that you can do it in a way that guarantees that it's
safe to add elements while iterating, or you're going to have to adjust what
you're doing so that it doesn't need to add to the list of items while
iterating over it.

The big issue with foreach is that if it's iterating over is a range, then
it copies it, and if it's not a range, it slices it (or if it defines
opApply, that gets used). So,

foreach(e; range)

gets lowered to

foreach(auto __c = range; !__c.empty; __c.popFront())
{
    auto e = __c.front;
}

which means that range is copied, and it's then unspecified behavior as to
what happens if you try to use the range after passing it to foreach (the
exact behavior depends on how the range is implemented), meaning that you
really shouldn't be passing a range to foreach and then still do anything
with it.

If foreach is given a container, then it slices it, e.g.

foreach(e; container)

foreach(auto __c = container[]; !__c.empty; __c.popFront())
{
    auto e = __c.front;
}

so it doesn't run into the copying problem, but it's still not a good idea
to mutate the container while iterating. What happens when you try to mutate
the container while iterating over a range from that container depends on
the container, and foreach in general isn't supposed to be able to iterate
over something while it's mutated.

Dynamic and associative arrays get different lowerings than generic ranges
or containers, but they're also likely to run into problems if you try to
mutate them while iterating over them.

So, if using a normal for loop instead of foreach fixes your problem, then
there you go. Otherwise, rearrange what you're doing so that it doesn't need
to add anything to the original list of items in the loop. Either way,
trying to mutate what you're iterating over is going to cause bugs, albeit
slightly different bugs depending on what you're iterating over.

- Jonathan M Davis