Dealing with the interior pointers bug

[H. S. Teoh via Digitalmars-d-learn]

On Wed, Jun 21, 2017 at 05:11:41PM +0000, TheGag96 via Digitalmars-d-learn wrote:
> On Wednesday, 21 June 2017 at 15:42:22 UTC, Adam D. Ruppe wrote:
> > This comes from the fact that D's GC is conservative - if it sees
> > something that *might* be a pointer, it assumes it *is* a pointer
> > and thus had better not get freed.
> 
> So is the GC then simply made to be "better-safe-than-sorry" or is
> this a consequence of how the GC does things? Or rather, does the GC
> know the type of any references to its memory at all?

The reason the GC must be conservative is because (1) D is a systems
programming language, and also because (2) D interfaces directly with C.

Being a systems programming language means you should be able to do
things outside the type system (in @system code, of course, not in @safe
code), including storing pointers as int values.  Any C code that your D
program interoperates with may also potentially do similar things.

Because of this, the GC cannot simply assume that an int value isn't
actually a pointer value in disguise, so if that int value happens to
coincide with an address of an allocated memory block, the only sane
thing it can do is to assume the worst and assume that the memory is
still live (via that (assumed) reference).  It's not safe for the GC to
assume that it's merely an int, because if it actually turns out to be a
pointer, then you'll end up with a dangling pointer and the ensuing
memory corruption, security holes, and so forth.  But assuming that the
value is a pointer is generally harmless -- the memory block just
doesn't get freed right away, but if the int is mutated afterwards,
eventually the GC will get around to cleaning it up.

The only big problem is in 32-bit code, where because of the very
limited space of pointer values, the chances of a random int value
coinciding with a valid pointer value is somewhat high, so if you have a
large allocated memory block, the chances of a random int being mistaken
for a reference to the block is somewhat high, so you could potentially
run out of memory due to large blocks not being freed when they could
be.  Fortunately, though, in 64-bit land the space of pointer values is
generally so large that it's highly unlikely that a random int would
look like a pointer, so this generally isn't a problem if you're using
64-bit, which is the case more and more now as vendors are slowly
phasing out 32-bit support.


T

-- 
People tell me that I'm skeptical, but I don't believe them.