Passing large or complex data structures to threads

[Joseph Rushton Wakeling]

On 05/27/2013 02:08 PM, Joseph Rushton Wakeling wrote:
> On 05/26/2013 05:59 PM, Ali Çehreli wrote:
>> On 05/26/2013 05:38 AM, Simen Kjaeraas wrote:
>>
>>
>>>      Tuple!(size_t, size_t)[][] data = createData();
>>>      immutable dataImm = assumeUnique(data);
>>>      data = null; // Simply to ensure no mutable references exist.
>>
>> The last line is not needed. assumeUnique already does that. :)
> 
> That's fantastic, thank you both very much.  Does that also work for arbitrary
> data structures (e.g. also associative arrays, complex structs/classes etc.)?
> 
> Related question -- assume that I now want to store that immutable data inside a
> broader storage class, but I want that storage class to be agnostic as to
> whether the data is immutable, const or mutable.

I thought I'd mention the practical solution I eventually came to here, which is
a combination of things learned from this discussion and the subsequent one on
duplicating a multidimensional array.

I first of all had a go at tweaking the templatization of my storage classes, in
line with some of the tricks Ali suggested for the multidimensional duplication
function.  In this case a storage class that had this form (cutting out
extraneous details):

	class SeedSIS(T)
	{
		T[] immune;
		T[] susceptible;
		T[] recover;
		Link!T[][] network;

		this(T[] imm, T[] sus, T[] rec, Link!T[][] net)
		{
			immune = imm;
			susceptible = sus;
			recover = rec;
			network = net;
		}
	}

... was tweaked to this form:

	class SeedSIS(T, Network : L[][], L)
	{
		T[] immune;
		T[] susceptible;
		T[] recover;
		Network network;

		this(T[] imm, T[] sus, T[] rec, Network net)
		{
			immune = imm;
			susceptible = sus;
			recover = rec;
			network = net;
		}
	}

... which enabled the network input to be mutable or immutable.  With a bit more
effort, I guess it would be possible to tweak the other inputs to extend that
option to them; it could be useful as a guarantee related to what bits of the
Seed class will change.

However, I found that in practice this reworked design (which involved reworking
lots of templates elsewhere as well) led to slowdown in some crucial bits of the
code.  So, instead I found an alternative solution following Ali's guidance
about using to!()() as a copying device, which was to pass the immutable network
to threads, and then take a mutable local copy via,

	mutableNetwork = to!(Link!T[][])(immutableNetwork);

... which could then be used with the original templates.