TempAlloc and druntime GC

[dsimcha]

== Quote from Sean Kelly (sean at invisibleduck.org)'s article
> == Quote from dsimcha (dsimcha at yahoo.com)'s article
> > I've been thinking about how to deal with this issue and make TempAlloc safe
> > enough for inclusion in the standard library.  Since it works by allocating
> > large (currently 4 MB) regions from the C heap, and then sub-allocating these
> > regions to its clients, simply scanning these whole chunks is simply not a
> > reasonable option, as it would lead to way too many false pointers.  Using the
> > GC's addRegion/removeRegion interface at every TempAlloc allocation is also
> > not a reasonable option, as this operation requires a lock and would negate
> > any performance gains from using TempAlloc instead of the GC heap.  The only
> > alternative I see is to make the GC aware of TempAlloc at a very low level.
> > I've looked at the GC code and this doesn't look terribly hard to do.  Using
> > some bit twiddling tricks, I can do the bookkeeping to determine which
> > TempAlloc allocated objects should be scanned by the GC w/o any additional
> > space overhead.
> The way I'd do this at the druntime level is generate a Thread.Context struct
> for each distinct block used by TempAlloc.  It shouldn't be too difficult to
> create an API to provide this functionality.  I'd have to do something roughly
> along these lines anyway to get thread support working for DLLs in Win32.

Can you elaborate on this idea?  I don't understand it, so I can't comment on
whether it would work.

> > The downside to this is that TempAlloc would be very tightly coupled to
> > druntime. Furthermore, if Sean and the other druntime maintainers decided they
> > didn't like my patch, my effort will be wasted.  Therefore, I'd like to get
> > some feedback before I bother doing anything.  Do you think that TempAlloc is
> > universally useful enough to belong in the standard library?  If so, do you
> > believe that it is only suitable for inclusion if it can be made safe w/
> > respect to storing the only reference to GC-allocated objects?  If so, is it
> > worth tightly coupling it to the GC, and should I start working on this and
> > submit a patch to druntime?
> For any new runtime features, the easiest thing is just to submit a ticket or
> to contact me and tell me what you need.  At first blush, I don't see any
> problem with providing the functions you'd need to make this work.
> It may eventually make sense to put TempAlloc, or something like it, into
> core.memory anyway (there's a reason I didn't name that module core.gc).
> Sean

It seems like you want to create some kind of more abstract interface to solve the
more general problem of interfacing stuff with the GC, rather than just
integrating TempAlloc directly at a low level.  One way this could be done (not
necessarily the best way, but the best I've thought of so far) is to make it
possible to register class objects conforming to some specific range interface (an
explicit runtime interface w/ virtual functions, not an implicit compile-time
interface, since we care about the ABI here) with the GC.  These objects would
return a pointer at each iteration, and then the GC would mark whatever was
pointed to by that pointer.  This would allow me to use some hacks to keep
TempAlloc efficient, while keeping these hacks well-encapsulated and out of the
core GC code.

In TempAlloc's case specifically, every time a new thread's TempAlloc state was
initialized, it would register one of these objects with the GC.  This object
would contain the thread's ID internally, and be set to return empty immediately
if the thread was already dead.  (TempAlloc-allocated data is absolutely NOT meant
to be shared between threads.)  Otherwise, it would scan through the currently
allocated elements and provide the GC with the pointers that are located there,
through the range interface.

I'm not sure if this would also work for DLL stuff since I know nothing about that
problem.  However, if this would solve the other cases you have in mind, I think
it's an elegant solution.