Threading and the Garbage handler mess.

[Alan Knowles]

Sergey Gromov wrote:
> Alan Knowles <alan at akbkhome.com> wrote:
>> "THE GC AS DELIVERED IS COMPLETELY BROKEN FOR REAL THREADED APPLICATONS
> 
> Isn't it a bit overstated?
Yes and no, the email was a good summary of all the things I was going 
through trying (and still am) random segfaults and large memory usage. - 
even by writing it down helped a lot in understanding the whole issue. 
(and I hope helps others a bit..)

I'm alot closer now to solving the issues than I think I was... In part 
by writing the email and explaining the issues to myself..

- But I still think that a large multi-threaded application having to 
rely on the GC is not a pretty prospect, and that being able to manage 
memory in a tighter clearer, and cleaner way would be preferable.. - 
imagine 500 threads being paused and scanned every few seconds to ensure 
memory doesn't get out of hand - this doesn't sound efficient... 
(althought I'm not sure I've got any great answers yet)

It's also a wake-up call to the authors of things like DBI - which 
really should read this and understand that you can not send connect() 
the username/password etc. without adding root to them.... - which I 
doubt currently many  libraries in dsource address..

> 
> You basically say that memory could be prematurely freed if an external 
> non-D function stores a pointer to GC-allocated memory internally.  This 
> is quite different from "COMPLETELY BROKEN" to me.  You don't even 
> mention that for both extern(C) and extern(Windows) pointers are pushed 
> on stack so that memory is protected at least until the external 
> function returns.  Well it *can* overwrite its arguments with an 
> unrelated data, but in all other scenarios your memory is protected.
> 
> The solutions are:
> 
> 1. Don't bother.  This is the correct approach for most of CRT and Win32 
> functions because they do not store pointers.
> 
> 2. Write wrappers.  If you must use a non-pure, non-reentrant C library 
> which stores pointers, or returns malloc'd memory,---then yes, you must 
> understand how GC works, how to protect GC-allocated memory, and how to 
> make GC handle externally-allocated memory.
> 
> 3. Write D-aware external functions.  If you implement them yourself you 
> can always copy memory instead of storing raw pointers, so that you can 
> follow solution 1 when writing D code.