GC, Multithreading, Heap(s)

[g012]

Le Wed, 5 Nov 2008 17:59:10 -0500, Steven Schveighoffer a écrit :

> "g012" wrote
>> Hello,
>>
>> I'm not yet programming seriously in D, but I've been lurking on it since
>> quite some years now, and I have some questions about its current (or
>> future ?) features related to memory that need answers before the decisive
>> jump.
>>
>> In the context of a multithreaded program, I'd like to know if it's
>> possible to do the following in D (if a feature is not currently 
>> supported,
>> can you indicate if it's planned for a future release):
>>
>> . replacing the default heap allocator with a multithread optimized one,
>> like nedmalloc. I saw I can overload new and delete in a class / struct,
>> but I'd like to do the overload on the generic new / delete. If it's not
>> possible, does the default allocator already support multithreaded heap
>> allocations without locking systematically ?
> 
> With the new druntime, the garbage collector is completely replacable (this 
> was always the case with Tango).  So you can do whatever you want ;)
> 
>> . is the GC multithreaded, ie when it pauses all the threads to run, does
>> it run multithreaded itself, with one thread per hardware thread ?
> 
> No
> 
>> . if a thread has data to be collected that only itself is referencing,
>> will a dedicated GC thread run in parallel of the other threads 
>> (themselves
>> running as normal, unpaused, simultaneously to the thread-local GC) ? If
>> no, is there a way to declare that a variable will be referenced only in
>> the current thread, and to have a thread-specific collector that does not
>> imply to pause other threads to run ? The idea is to collect thread-local
>> variables by a thread-locale GC, and shared variables by the global GC. 
>> And
>> if it is possible, can we disable temporarily thread-local GC similarly to
>> the global GC ?
>>
>> If anything that can optimize heap usage in threads better than the above
>> points is already implemented (or planned to) in D, please explain it to 
>> me
>> :)
> 
> See this article on a future enhancement planned for D.  Specifically make 
> sure you read the last paragraph.
> 
> http://bartoszmilewski.wordpress.com/2008/07/30/sharing-in-d/
> 
> -Steve

Hey,

Thanks for the quick reply !

Indeed, the article is very interesting. The last paragraph describes
exactly what I'm doing with Lua (but what he suggests is more efficient,
since it's a language feature). I've got some moddable program, in C, which
is modded by Lua. I have 3 main threads running in C, each of them allocate
a Lua state, and there is a global Lua state for shared data. So a mod has
to provide 3 Lua scripts to access all threads, one for each. Whenever a
shared data is to be accessed, it's going (via a metatable function call)
to locked C code which copies / reads the data to / from the shared Lua
state. Of course each Lua state has its own data, its own GC, and its own
heap (I create a small private heap for each Lua state, so they can't
create memory overflow on the main heap or abuse memory allocation). So the
result is pretty much what's described in that last paragraph, with the
exception of much less transparency and copy overhead on the programmer
side since it's not a Lua feature but a C implementation.

I hope it gets adopted (even if I have no idea how shared-casting can be
implemented efficiently !), and implemented.

Pascal