How to resume iteration from previous point in associative array

[simendsjo]

On Wednesday, 19 February 2014 at 09:21:48 UTC, Tarman wrote:
> Hi,
>
> We're doing some "super computing" "big data" style stuff with 
> D. We have a system where we're comparing associative arrays 
> with billions of entries.
>
> However in this system we need to fairly consider possible 
> solutions for many "units" at a time within a single thread.
>
> So we'd like to... say, first iterate over the first 10 million 
> for each "unit" then iterate over the next 10 million for the 
> next unit so that each unit gets a fair share of CPU time.
>
> However in this case we can't:
>
> int count = 0;
> foreach (item; associativeArray) {
>   if (++count == 10_000_000) {
>     // here we wish to somehow save the iteration state
>   }
> }
>
> Then on the next iteration:
>
> foreach (resume from previous iteration point) {
> }
>
> We've tried copying the keys into a non-associative array and 
> sure this works, but it is far far far less optimal than an 
> equivalent C++ solution we wrote where we use an 
> std::unordered_set and can simply store the iterator.

Probably not an answer to your question, but if possible, you can 
use a fiber and yield when you should give control to another 
fiber.
http://dlang.org/phobos/core_thread.html#.Fiber