Modify thread-local storage from parent thread

[Kai Meyer]

On 08/09/2011 10:27 AM, Steven Schveighoffer wrote:
> On Tue, 09 Aug 2011 11:36:13 -0400, Kai Meyer <kai at unixlords.com> wrote:
>
>> On 08/08/2011 01:38 PM, Steven Schveighoffer wrote:
>>> On Mon, 08 Aug 2011 14:17:28 -0400, Kai Meyer <kai at unixlords.com> wrote:
>>>
>>>> I am playing with threading, and I am doing something like this:
>>>> file.rawRead(bytes);
>>>> auto tmpTask = task!do_something(bytes.idup);
>>>> task_pool.put(tmpTask);
>>>> Is there a way to avoid the idup (or can somebody explain why idup
>>>> here is not expensive?)
>>>
>>> I'd have to see where bytes is created, if it's created in the same
>>> context, just casting to immutable is allowed, as long as you never use
>>> the mutable reference again.
>>>
>>>> If the logic above is expressed as:
>>>> Read bytes into an array
>>>> Create a thread (task) to execute a function that takes a copy of
>>>> 'bytes'
>>>> Execute the thread
>>>>
>>>> I wonder if I could:
>>>> Create a thread (task)
>>>> Read bytes directly into the tasks' thread local storage
>>>> Execute the thread
>>>
>>> This *might* be possible. However, in many cases, the OS is responsible
>>> for creating the TLS when the thread starts, so you have to wait until
>>> the thread is actually running to access it (not an expert on this, but
>>> I think this is the case for everything but OSX?)
>>>
>>> So you would have to create the thread, have it pause while you fill
>>> it's TLS, then resume it.
>>>
>>> But I think this is clearly a weird approach to this problem. Finding a
>>> way to reliably pass the data to the sub-thread seems more appropriate.
>>>
>>> BTW, I've dealt with having to access other threads' TLS. It's not
>>> pretty, and I don't recommend using it except in specialized situations
>>> (mine was adding a GC hook).
>>>
>>> -Steve
>>
>> Well, bytes is in a loop, so casting to immutable wouldn't do it. The
>> idea is to read a block of bytes, and hand them off to a worker thread
>> to operate on those set of bytes. Everything is working, I'm just
>> trying to avoid having to reallocate that block of bytes for the read,
>> and then reallocate them again to pass them off to the worker thread.
>> If I could get away with one allocation, I'd be happier.
>
> OK, there are other options. First, you could keep a "pool" of buffers,
> which are marked as shared. When you want to run a task, get one of
> those buffers, fill it, then pass the buffer to the task thread to
> process. Make sure the task thread puts the buffer back into the pool
> when it's done. I'd recommend casting the buffer to unshared while
> inside the task thread to save some cycles. This is probably the option
> I'd go with.
>
> Second, you can have the task thread give you it's TLS buffer to read
> data into (you need to do some casting to get this around the type
> system). Note that in order for it truly to be stored in TLS, the buffer
> has to be a fixed-sized array.
>
> -Steve

These are concepts that I'm only familiar with. I think I would like to 
try the "pool" of buffers. I can't say I know how to mark buffers as 
shared, though. Could you modify this for me, to show me an example?

import std.parallelism;
ubyte[8][] pool; // Dynamic array of (array of 8 bytes)
void thread_func()
{
     //my_pool[0] = 50;
}
void main(string[] args)
{
     uint threads = 2;
     pool.length = threads;
     TaskPool taskpool = new TaskPool(threads);
     foreach(i; 0..threads)
     {
         auto tmpTask = task!thread_func();
         taskpool.put(tmpTask);
     }
     taskpool.stop();
}