Memory Management in D: Request for Comment

[Leandro Lucarella]

dsimcha, el  3 de noviembre a las 05:46 me escribiste:
> During my occasional forays into the dark side of Python and Java, I am often
> amazed at the extent to which memory management in these languages "just
> works".  D should be like this for all but the most low-level programming
> tasks, and was intended to be.  It seems like most of the other regulars here
> have carved out niches that don't involve improving memory management.
> 
> My attempts at adding precise heap scanning to the GC got me thinking about
> other ways to improve memory management in D.  I love most aspects of D, but
> the constant memory management issues make it feel like much less of a
> high-level language than it should feel like.  I'm thinking of making this my
> niche around here, as I already know more about the problem than I ever wanted
> to know and I'm sick of having memory management not work properly.  Here are
> some things that I'd like comments on:
> 
> 1.  In the Hans Boehm GC, they use a blacklisting scheme whereby they avoid
> allocating memory pages that currently have false pointers pointing into them.
>  (If a page is not allocated but looks like it has a pointer into it, then we
> can assume this is a false pointer.)  If I dip into the GC code again and
> implement something like this, we'll be one step closer to making D memory
> management "just work" and making false pointers a thing of the past.

The Boehm GC has a lot to learn from. I'm planning to add concurrent
collection in a sightly different way than the Boehm GC, which have at
least a couple of ways of doing so. I know it might look like vaporware at
this point but I have to do it eventually, I need to finish that to
finally get my degree ;). I was just a little busy lately.

> 3.  This one is an order of magnitude less likely than the other two to
> actually get implemented, at least by me, but how about thread-local
> allocators so you can call malloc() without taking a lock?  I vaguely remember
> Sean saying he was working on that a while back, but I never heard anything
> about it again.  It's probably best to wait for shared to be implemented for
> this so that unshared objects can also be collected w/o stopping the world,
> but we should start at least discussing this now.

I think avoiding the lock might be possible using atomic ops. I think it
should be easier than using thread-local, but I'm not entirely sure if
it's really possible.

> 4.  I submitted a patch a while back to allow the GC to ignore interior
> pointers for specific objects.
> (http://d.puremagic.com/issues/show_bug.cgi?id=2927)  This would be useful if
> you have, for example, a large array that never escapes a class and the class
> always maintains a pointer to the head of the array as long as it's alive.
> This way, when the class dies, the array dies too even if there are false
> pointers to its interior.  Few people have commented on this.  Is there any
> reason why it's not a good idea?  Yes, it's somewhat unsafe if you're not
> careful, but when the alternative is horrible memory leaks, sometimes
> unsafeness is a necessary evil.

I think I commented it, and I think is a good idea to have it. It's not
unsafer than having NO_SCAN or casting a pointer to a size_t.

-- 
Leandro Lucarella (AKA luca)                     http://llucax.com.ar/
----------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------
Todos en el mundo somos grasas, no hago distinción de sexo y raza, sólo
que algunos lo disfrutan y otros no pueden evitarlo.