Passing large or complex data structures to threads

[Ali Çehreli]

On 05/24/2013 06:26 AM, Joseph Rushton Wakeling wrote:

 > Are there any recommended strategies for passing large or complex data
 > structures (particularly reference types) to threads?

std.concurrency works with shared data.

 > For the purpose of this discussion we can assume that it's read-only 
 > data

The following simple example uses mutable data but it should work with 
'const' too.

import std.stdio;
import std.concurrency;
import std.typecons;
import core.thread;

alias DataElement = Tuple!(size_t, size_t);
alias DataRow = DataElement[];
alias Data = DataRow[];

enum size_t totalRows = 4;

void func(Tid owner, shared(Data) data, size_t rowId)
{
     foreach (ref element; data[rowId]) {
         element[0] *= 10;
         element[1] *= 10;
     }
}

shared(Data) makeData()
{
     shared(Data) data;

     foreach (size_t row; 0 .. totalRows) {
         shared(DataRow) dataRow;

         foreach (size_t col; 0 .. 10) {
             dataRow ~= tuple(row, col);
         }

         data ~= dataRow;
     }

     return data;
}

void main()
{
     shared(Data) data = makeData();

     writeln("before: ", data);
     foreach (rowId, row; data) {
         // Instead of 'data' and 'rowId', the child could take
         // its own row (not tested)
         spawn(&func, thisTid, data, rowId);
     }

     thread_joinAll();
     writeln("after : ", data);
}

Ali