Changes to the Tango runtime / GC

[Sean Kelly]

All the recent talk about the GC inspired me to make some changes that 
I'd been planning for Tango.  I wanted to post about them here to make 
sure everyone was aware of them because they are silent changes that 
could affect the reliability of a program in certain rare circumstances. 
  Previously, the Tango runtime worked just like the Phobos runtime in 
that the hasPointers flag was set or cleared on array operations based 
on the type of the variable referencing the underlying memory.  For example:

byte[] b = cast(byte[]) new void[5]; // hasPointers is set
byte[] c = b[1 .. $];

b.length = 10; // hasPointers is retained because no realloc occurs
b.length = 20; // hasPointers is lost/cleared because of a realloc

c.length = 1;  // hasPointers is lost/cleared--slices always realloc
c.length = 20; // hasPointers is lost/cleared for same reason as above

The opposite was also true:

void[] v = new byte[5]; // hasPointers is not set
void[] w = v[1 .. $];

v.length = 10; // hasPointers is still 0 because no realloc occurs
v.length = 20; // hasPointers is set because of a realloc

w.length = 1;  // hasPointers is set because slices always realloc
w.length = 20; // hasPointers is set for same reason as above

The new behavior of Tango is to preserve block attributes for any 
allocated block whose size is simply changing as the result of an array 
operation.  *** This is true of both slices and normal arrays. ***  For 
example:

byte[] b = cast(byte[]) new void[5]; // hasPointers is set
byte[] c = b[1 .. $];

b.length = 10; // hasPointers is retained because no realloc occurs
b.length = 20; // hasPointers is retained because of new logic

c.length = 1;  // hasPointers is retained because of new logic
c.length = 20; // hasPointers is retained because of new logic

The same behavior is true of void references to byte arrays.  Here is a 
quick run-down of more complex cases:

byte[] b = cast(byte[]) new void[5];

b ~= 0;          // hasPointers is preserved on realloc
b ~= [0,1,2];    // hasPointers is preserved on realloc
b = [0,1] ~ [2]; // hasPointers is not retained - A

b = null;
b.length = 10;   // hasPointers is lost because reference was cleared

In situation A above, hasPointers is not retained because the operation 
is an assignment rather than a resize.

I believe these changes will result in more predictable behavior than 
before.  However, they can cause a change in program behavior in rare cases:

void[] getBlock()
{
     return new byte[32];
}

With the old behavior, manipulating the block returned by getBlock in a 
way that caused a reallocation to occur would result in hasPointers 
being set on that block.  With the new behavior, hasPointers would not 
be set because the runtime would be preserving the bits set on the 
original block, which was allocated as a byte[].  Thus it's possible in 
certain degenerate cases that the bits set on a memory block could 
silently propagate and "poison" an application in instances where they 
were discarded before.


Sean