Some GC and emulated TLS questions (GDC related)

[Johannes Pfau via Digitalmars-d]

Am Sat, 15 Jul 2017 10:49:39 +0000
schrieb Joakim <dlang at joakim.fea.st>:

> On Friday, 14 July 2017 at 09:13:26 UTC, Johannes Pfau wrote:
> > Another solution could be to enhance libgcc emutls to allow 
> > custom allocators, then have a special allocation function in 
> > druntime for all D emutls variables. As far as I know there is 
> > no GC heap that is scanned, but not automatically collected?  
> 
> I believe that's what's done with the TLS ranges now, they're 
> scanned but not collected, though they're not part of the GC heap.

Indeed. We used to use GC.addRange for this and this was said to be
slow when using many ranges. So I'm basically asking whether the scan
delegate has got the same problem or whether it can cope with thousands
of small ranges.
A scanned but not collected heap is slightly different, as the GC can
internally treat the allocator memory as one huge memory range. When
allocating using C malloc, every single allocation needs to be scanned
individually. A scan+/do not collect allocator can probably be built
using the std.experimental.allocator primitives but that code is not in
druntime.

> 
> > I'd need a way to completely manually manage GC.malloc/GC.free 
> > memory without the GC collecting this memory, but still 
> > scanning this memory for pointers. Does something like this 
> > exist?  
> 
> It doesn't have to be GC.malloc/GC.free, right?  The current 
> DMD-style emutls simply mallocs and frees the TLS data itself and 
> only expects the GC to scan it.

The problem here again is whether this scales properly when using
thousands of non contiguous memory ranges. DMD style TLS can allocate
one memory block per thread for all variables. GCC style will allocate
one block per thread and variable.

> 
> > Another option is simply using the DMD-style emutls. But as far 
> > as I can see the DMD implementation never supported dynamic 
> > loading of shared libraries? This is something the GCC emutls 
> > support is quite good at: It doesn't have any platform 
> > dependencies (apart from mutexes and some way to store one 
> > thread specific pointer+destructor) and should work with all 
> > kinds of shared library combinations. DMD style emutls also 
> > does not allow sharing TLS variables between D and other 
> > languages.  
> 
> Yes, DMD's emutls was never made to work with loading multiple 
> shared libraries.  As for sharing with other languages without 
> copying the TLS data over, that seems a rare scenario.

Yes, probably the best solution for now is to reimplement GCC style
emutls with shared library support in druntime for all compilers and
forget about C/C++ TLS compatibility. Even if we could get patches into
libgcc it'd take years till all relevant systems have been updated to
new libgcc versions.

> 
> > So I was thinking, if DMD style emutls really doesn't support 
> > shared libraries, maybe we should just clone a GCC-style, 
> > compiler and OS agnostic emutls implementation into druntime? A 
> > D implementation could simply allocate all internal arrays 
> > using the GC. This should be just as efficient as the C 
> > implementation for variable access and interfacing to the GC is 
> > trivial. It gets somewhat more complicated if we want to use 
> > this in betterC though. We also lose C/C++ compatibility though 
> > by using such a custom implementation.  
> 
> It would be a good alternative to have, and you're not going to 
> care in betterC mode, since there's no druntime or GC.  You'd 
> have to be careful how you called TLS data from C/C++, but it 
> could still be done.
> 
> > The rest of this post is a description of the GCC emutls code. 
> > Someone
> > can use this specification to implement a clean-room design D 
> > emutls
> > clone.
> > Source code can be found here, but beware of the GPL license:
> > https://github.com/gcc-mirror/gcc/blob/master/libgcc/emutls.c
> >
> > [...]  
> 
> There is also this llvm implementation, available under 
> permissive licenses and actually documented somewhat:
> 
> https://github.com/llvm-mirror/compiler-rt/blob/master/lib/builtins/emutls.c

Unfortunately also not boost compatible, so we can't simply port that
code either, as far as I can see?


-- Johannes