How to resume iteration from previous point in associative array

Tarman tarman at chilon.net
Wed Feb 19 01:21:47 PST 2014


Hi,

We're doing some "super computing" "big data" style stuff with D. 
We have a system where we're comparing associative arrays with 
billions of entries.

However in this system we need to fairly consider possible 
solutions for many "units" at a time within a single thread.

So we'd like to... say, first iterate over the first 10 million 
for each "unit" then iterate over the next 10 million for the 
next unit so that each unit gets a fair share of CPU time.

However in this case we can't:

int count = 0;
foreach (item; associativeArray) {
   if (++count == 10_000_000) {
     // here we wish to somehow save the iteration state
   }
}

Then on the next iteration:

foreach (resume from previous iteration point) {
}

We've tried copying the keys into a non-associative array and 
sure this works, but it is far far far less optimal than an 
equivalent C++ solution we wrote where we use an 
std::unordered_set and can simply store the iterator.


More information about the Digitalmars-d-learn mailing list