AddressSanitizer and the GC

[Johan Engelen via Digitalmars-d]

Hi all,
   I've added AddressSanitizer [1] support to LDC, and it is able 
to nicely catch bugs like this one:
```
void foo(int* arr) {
     arr[10] = 1;
}

void main() {
     int[10] a;
     foo(&a[0]);
}
```

It works for malloc'ed memory, but not yet for GC'd memory. Our 
GC pool memory is malloc'ed, so it is "safe" to access from 
ASan's current point of view. To make it work with our GC, we 
have to mark all GC pool memory (malloc'ed) as poisoned, such 
that access to that memory will trigger an ASan error. Then when 
a user asks memory from the pool (e.g. `new int`), we should 
allocate a little more GC'd memory than what the user requests 
(red zone), and then mark only the user size as unpoisoned. By 
allocating a little more, we make sure that user allocated memory 
blocks are separated by at least an amount of red zone that helps 
detect read/write memory overflows.
Pieces of the solution:
a. Call __asan_poison_memory_region on all memory internally 
allocated by the GC
b. ptr = GC.alloc(user_request_size+redzone)
c. Call __asan_unpoison_memory_region on ptr[0..user_request_size]

I've already been toying with several options but would like your 
opinion: what is the best way for me to add this to the GC?

1. What are the best places to modify the conservative GC with 
the above pieces?

2. Should I create a new GC type "asan"? We now have 
"conservative" (default) and "manual". Adding an option to the 
default GC is made hard because of linking problems (need to link 
with asan library, but I can define @weak function stubs), but I 
am worried about slow down in non-asan mode (which is the 
99.9999999% use case).

Thanks,
   Johan


[1] https://github.com/google/sanitizers/wiki/AddressSanitizer