AddressSanitizer and the GC

[Johan Engelen via Digitalmars-d]

On Friday, 21 July 2017 at 20:53:18 UTC, Walter Bright wrote:
> On 7/21/2017 12:06 PM, Johan Engelen wrote:
> < [...]
>
> Thanks for your work on this. It is important to support such 
> valuable tools.
>
> One way making such things pretty much cost-free in the olden 
> days was to trick the linker into writing NOPs over the 
> function calls by using specially crafted fixup records. This 
> worked with dumb linkers, but today's linkers do much more 
> semantic inference into what's going on, I'm afraid this may be 
> too brittle.
>
> Unless there is a linker feature to support it? Are some 
> linkers able to NOP out a function call if it sees that the 
> function body consists solely of a RET?

ARM's toolchain has something that does this:
http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.faqs/ka14171.html

Can't find something like it for other toolchains.

> Another option is to use a pointer to the asan functions, like:
>
>     if (asan_fp)
>         (*asan_fp)(args...)
>
> The asan_fp can then be set to NULL and there's only the 
> overhead of a compare/branch rather than a function call. Well, 
> also the load of a global variable.

I can mark the if-clause as unlikely, so the performance 
difference would be small (allocation is slow already, of course).
Let me try that for now. We can decide later, let's first find an 
implementation that works and test that the whole idea is valid 
and I didn't miss anything important.

Can you advise me on where best to add the asan calls?

Thanks,
   Johan

PS. The interface is easy, I don't need much info at the call 
sites:
// Poisons memory region [addr, addr+size)
void __asan_poison_memory_region(const(void*) addr, size_t size);
// Unpoisons memory region [addr, addr+size)
void __asan_unpoison_memory_region(const(void*) addr, size_t 
size);