Reserving/Preallocating associative array?

[Andrei Alexandrescu]

On 12/24/13 2:28 PM, Gordon wrote:
> Hello,
>
> I want to load a large text file containing two numeric fields into an
> associative array.
> The file looks like:
>     1   40
>     4   2
>     42  11
>     ...
>
> And has 11M lines.
>
> My code looks like this:
> ===
> void main()
> {
>          size_t[size_t] unions;
>          auto f = File("input.txt");
>          foreach ( line ; f.byLine() ) {
>                  auto fields = line.split();
>                  size_t i = to!size_t(fields[0]);
>                  size_t j = to!size_t(fields[1]);
>                  unions[i] = j; // <-- here be question
>          }
> }
> ===
>
> This is just a test code to illustrate my question (though general
> comments are welcomed - I'm new to D).
>
> Commenting out the highlighted line (not populating the hash), the
> program completes in 25 seconds.
> Compiling with the highlighted line, the program takes ~3.5 minutes.
>
> Is there a way to speed the loading? perhaps reserving memory in the
> hash before populating it? Or another trick?

void main()
{
     size_t[size_t] unions;
     foreach (e; "input.txt"
             .slurp!(size_t, size_t)("%s %s").sort.uniq ) {
         unions[e[0]] = e[1];
     }
}


Andrei