How to resume iteration from previous point in associative array

[Tobias Pankrath]

On Wednesday, 19 February 2014 at 09:21:48 UTC, Tarman wrote:
> Hi,
>
> We're doing some "super computing" "big data" style stuff with 
> D. We have a system where we're comparing associative arrays 
> with billions of entries.
>
> However in this system we need to fairly consider possible 
> solutions for many "units" at a time within a single thread.
>
> So we'd like to... say, first iterate over the first 10 million 
> for each "unit" then iterate over the next 10 million for the 
> next unit so that each unit gets a fair share of CPU time.
>
> However in this case we can't:
>
> int count = 0;
> foreach (item; associativeArray) {
>   if (++count == 10_000_000) {
>     // here we wish to somehow save the iteration state
>   }
> }
>
> Then on the next iteration:
>
> foreach (resume from previous iteration point) {
> }
>
> We've tried copying the keys into a non-associative array and 
> sure this works, but it is far far far less optimal than an 
> equivalent C++ solution we wrote where we use an 
> std::unordered_set and can simply store the iterator.

You could iterate over byValue/byKey/zip(byKey,byValue) which 
gives you a range to start again from. You'd need to iterate by 
hand (with while,for) though, since your range won't get consumed 
by foreach, I guess. Or use RefRange.

not tested:

auto values = refRange(&associativeArray.byValue);
foreach(item; values)
{
   if(someday) break;
}

// here values should have been advanced accordingly
// and you can just continue your iteration
foreach(item; values)
{

}