Pay as you go is really going to make a difference

[Gregor Mückl]

On Sunday, 26 January 2020 at 00:18:49 UTC, Steven Schveighoffer 
wrote:
> On 1/25/20 6:41 PM, norm wrote:
>> On Saturday, 25 January 2020 at 16:07:24 UTC, Steven 
>> Schveighoffer wrote:
>>> On 1/24/20 5:20 PM, Gregor Mückl wrote:
>>>
>>>> So there's a couple of calls that should be fairly fast 
>>>> (clock_getres etc.). Why does the runtime need to read 
>>>> /proc/self/maps, though?
>>>
>>> This is great stuff, thanks!
>>>
>>> Note that /proc/self/maps appears nowhere in phobos or 
>>> druntime. So I'm assuming that's some other function that's 
>>> doing it (maybe in libc?)
>>>
>>> The clock_getres calls are so you can use core.time.MonoTime. 
>>> rt_init initializes those so they can be used early in the 
>>> process. I would assume those are really fast, as they are 
>>> constants in the kernel.
>>>
>> 
>> I was curious so I ran a quick strace hello world experiment 
>> using only printf, not writeln, compiled with D (DMD), C++ and 
>> C (gcc and clang). Only the D binary opens /proc/self/maps. 
>> Running `strace dmd --version` also opens /proc/self/maps, but 
>> I guess that makes sense since the compiler itself is now 
>> written in D.
>> 
>
> Yeah, it's not being opened directly by druntime, but looks 
> like pthread_getattr_np (man strace -k is useful!):
>
> openat(AT_FDCWD, "/proc/self/maps", O_RDONLY|O_CLOEXEC) = 3
>  > /lib/x86_64-linux-gnu/libc-2.27.so(__open_nocancel+0x41) 
> [0x10fdb1]
>  > /lib/x86_64-linux-gnu/libc-2.27.so(_IO_file_fopen+0x78d) 
> [0x8cc3d]
>  > /lib/x86_64-linux-gnu/libc-2.27.so(fopen+0x7a) [0x7eeaa]
>  > 
> /lib/x86_64-linux-gnu/libpthread-2.27.so(pthread_getattr_np+0x193) [0x95b3]
>  > /mnt/hgfs/Documents/testd/teststrace(thread_init+0x250) 
> [0x50d98]
>  > /mnt/hgfs/Documents/testd/teststrace(rt_init+0x4c) [0x3f880]
>  > 
> /mnt/hgfs/Documents/testd/teststrace(_D2rt6dmain212_d_run_main2UAAamPUQgZiZ6runAllMFZv+0x14) [0x3c55c]
>  > 
> /mnt/hgfs/Documents/testd/teststrace(_D2rt6dmain212_d_run_main2UAAamPUQgZiZ7tryExecMFMDFZvZv+0x21) [0x3c4f9]
>  > /mnt/hgfs/Documents/testd/teststrace(_d_run_main2+0x22e) 
> [0x3c462]
>  > /mnt/hgfs/Documents/testd/teststrace(_d_run_main+0xbe) 
> [0x3c21e]
>  > /mnt/hgfs/Documents/testd/teststrace(main+0x22) [0x3c136]
>  > /lib/x86_64-linux-gnu/libc-2.27.so(__libc_start_main+0xe7) 
> [0x21b97]
>  > /mnt/hgfs/Documents/testd/teststrace(_start+0x2a) [0x3c01a]
>
>
> So it's something to do with thread_init calling 
> pthread_getattr_np. I don't see a direct call in there, so 
> probably it's inlined or tail-calling.
>
> And is that really using fopen? my goodness..
>
> I'm not sure we can pay-as-you-go the low-level thread support. 
> And we can't do anything about how pthreads use the OS to 
> implement their mechanics.
>
> -Steve

You got some actually useful backtraces. Great! Looks like it is 
this implementation or very close to it:

https://code.woboq.org/userspace/glibc/nptl/pthread_getattr_np.c.html

So this opens the file if it doesn't have stack layout 
information readily available. If I read the code correctly, this 
information read form the file isn't even cached for the specific 
thread. So unless I'm missing something (and I hope I do...), 
pthread_getattr_np goes through that dance each and every time. 
Something about this feels odd.

I found two other references to /proc/self/maps, one of them as 
part of vfprintf when _FORTIFY_SOURCE is enabled (then the format 
string *must* be in read only page) and the other one in fatal 
error handling. Those shouldn't concern us.