TLS for Android

[Joakim]

On Sunday, 9 March 2014 at 16:12:19 UTC, David Nadlinger wrote:
> On 9 Mar 2014, at 8:36, Joakim wrote:
>> You mention "replacing the part that Glibc does (but Bionic 
>> doesn't) with a piece of code in druntime."  Just to be clear, 
>> you're referring to accessing TLS variables using an offset 
>> into the initialization image, which is what ___tls_get_addr 
>> from druntime does in Walter's packed TLS approach, right?  If 
>> not, I'm not sure exactly what you're referring to.  With all 
>> this TLS stuff split up between the compiler, linker, and 
>> runtime linker, often undocumented or poorly documenented in 
>> the latter two cases, it's been confusing to follow the TLS 
>> code path to see what's happening.
>
> There are several possible ABIs for thread-local storage. For 
> the sake of this argument, let's assume that our particular 
> system works like the Linux/x86 implementation or Walter's OS X 
> approach in that the TLS storage area is simply a flat block of 
> memory where the individual variables reside at some offset. 
> Then, there is still the question of how the application knows 
> a) the base address of the block and b) the offset of the 
> variable of interest.
>
> In Walter's OS X implementation, both is taken care of by 
> __tls_get_addr, which expects a pointer into the section where 
> the TLS initialization data is stored. On e.g. Linux/x86_64, 
> however, the base address is stored in %fs, and the offset is 
> provided by special linker relocations (which essentially 
> evaluate to the offset of a given symbol from the beginning of 
> the initialization image). No extra function calls are inserted 
> by the compiler here to access TLS data, and the (C) runtime is 
> not directly involved for the accesses.
>
> For an overview of the different models, see 
> http://www.akkadia.org/drepper/tls.pdf (which is the most 
> comprehensive document I could find, in spite of what you might 
> think about the author).
Yeah, I've had that pdf loaded in my browser for the last couple 
months, skimmed some of it initially and I've been slowly going 
through it in more detail.  I tried simply loading a binary built 
using bracketed sections and the linker's current TLS 
relocations, ie no extra function calls, in Android/x86 and I got 
some other random data in the resulting TLS initialization image. 
  I think this is because bionic stores the 
pthread_setspecific-created void* pointers in the normal TLS 
area, so you can't just use the TLS relocations that dmd and the 
gold linker generate for linux/x86 on Android/x86, ie using the 
%gs register directly.

I have no opinion on the author, should I? ;)

> But regardless of what model is chosen, there is still the 
> issue of actually setting up a copy of the data for each thread 
> during initialization. This is what I was referring to when I 
> mentioned "replacing the part that Glibc does (but Bionic 
> doesn't) with a piece of code in druntime".

I was finally able to access a proper initialization image 
created by dmd in druntime on Android/x86 a couple hours back, by 
using dl_phdr_info similarly to what is done on linux now.

> So, if %gs works as expected on Android and the linker supports 
> the necessary relocations, then it might be an option to simply 
> use the existing TLS implementation in LLVM and simply provide 
> the missing bits in druntime. On the other hand, if you choose 
> to go with an entirely different TLS scheme (such as the DMD OS 
> X implementation), you need to figure out how to change the 
> codegen to emit the extra function calls to your __tls_get_addr 
> analog, etc. Looking at 
> llvm/lib/Target/ARM/ARMISelLowering.cpp, there might actually 
> be a working implementation for this in LLVM already (which I 
> didn't realize before), so this route would not necessarily be 
> more complex than going with a different scheme. You'd probably 
> just need to provide the __tls_get_addr implementation in 
> druntime and figure out how LLVM emits the TLS image resp. how 
> to get its base address.
I think this is the best route, with the advantage that if my 
___tls_get_addr uses pthread_(get|set)specific, it will likely 
just work on ARM too.  I thought I'd have to get ldc to generate 
slightly different IR to do this, but it'd be great if llvm 
already does this.  I had briefly looked at X86ISelLowering.cpp 
but not the ARM one, I'll see what it does.

> Hope this helps,
> David
Yeah, I think we're on the same page, thanks for the explanation. 
  I've just been learning about TLS recently, so I wasn't sure 
before.