How to resume iteration from previous point in associative array
Tobias Pankrath
tobias at pankrath.net
Wed Feb 19 01:41:10 PST 2014
On Wednesday, 19 February 2014 at 09:21:48 UTC, Tarman wrote:
> Hi,
>
> We're doing some "super computing" "big data" style stuff with
> D. We have a system where we're comparing associative arrays
> with billions of entries.
>
> However in this system we need to fairly consider possible
> solutions for many "units" at a time within a single thread.
>
> So we'd like to... say, first iterate over the first 10 million
> for each "unit" then iterate over the next 10 million for the
> next unit so that each unit gets a fair share of CPU time.
>
> However in this case we can't:
>
> int count = 0;
> foreach (item; associativeArray) {
> if (++count == 10_000_000) {
> // here we wish to somehow save the iteration state
> }
> }
>
> Then on the next iteration:
>
> foreach (resume from previous iteration point) {
> }
>
> We've tried copying the keys into a non-associative array and
> sure this works, but it is far far far less optimal than an
> equivalent C++ solution we wrote where we use an
> std::unordered_set and can simply store the iterator.
You could iterate over byValue/byKey/zip(byKey,byValue) which
gives you a range to start again from. You'd need to iterate by
hand (with while,for) though, since your range won't get consumed
by foreach, I guess. Or use RefRange.
not tested:
auto values = refRange(&associativeArray.byValue);
foreach(item; values)
{
if(someday) break;
}
// here values should have been advanced accordingly
// and you can just continue your iteration
foreach(item; values)
{
}
More information about the Digitalmars-d-learn
mailing list