[Issue 17957] New: D shared library throws asserts when called from C detached pthread but not terminated with dlclose

[d-bugmail at puremagic.com]

https://issues.dlang.org/show_bug.cgi?id=17957

          Issue ID: 17957
           Summary: D shared library throws asserts when called from C
                    detached pthread but not terminated with dlclose
           Product: D
           Version: D2
          Hardware: x86_64
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P1
         Component: druntime
          Assignee: nobody at puremagic.com
          Reporter: ajidala at gmail.com

Okay, this is a complex one.

Say you have a shared library written in D which exports the symbol test_fun.
Now in a C application, you create a thread, which you detach, and then load
the shared library and execute test_fun. Meanwhile, the main thread waits for
the auxiliary thread to complete with pthread_join, and then simply terminate
the application.

This will lead to the D runtime throwing asserts, and the default assert
handler attempts to allocate memory with the GC, which makes the application
segfault because the GC is not enabled at this stage and the asserts happen in
a @nogc application anyway. If we replace the assert handler with something
that doesn't do GC allocations, we can see two places in the D runtime that
throw asserts; attempts to lock and unlock a pthread mutex.

Interestingly, if one calls dlclose() after executing test_fun, the application
terminates without asserts. Even more interestingly, using dlopen with the
RTLD_NODELETE flag, even dlclose() can't save us from druntime's assertiveness.

Here's an example application:

main.c:

#include <stdio.h>
#include <stdlib.h>
#include <dlfcn.h>
#include <pthread.h>

int run_lib() {
    printf("run_lib()\n");
    void *lib = dlopen("./libdtest.so", RTLD_NOW | RTLD_LOCAL);
    if (!lib) {
        fprintf(stderr, "dlopen failed: '%s'\n", dlerror());
    }

    int (*fun)() = dlsym(lib, "test_fun");
    char *error = dlerror();
    if (error) {
        fprintf(stderr, "dlsym failed: '%s'\n", error);
        exit(1);
    }
    (*fun)();
    // uncomment for no asserts!
    // dlclose(lib);

    return 0;
}

static void *object_thread(void *p) {
    pthread_detach(pthread_self());
    run_lib();
}

int main() {
    printf("main()\n");
    pthread_t thread;
    pthread_create(&thread, NULL, object_thread, NULL);
    pthread_join(thread, NULL);
    printf("main() done\n");
    fflush(stdout);
    return 0;
}


dtest.d:

import core.stdc.stdio;
import core.exception;

void dumb_assert_handler(string file, ulong line, string msg) nothrow {
    printf("Someone in %s on line %d is complaining.\n", file.ptr, line);
}

extern (C) int test_fun() {
    assertHandler(&dumb_assert_handler);
    printf("can confirm this is fun\n");
    return 0;
}


build.sh:

rm -rf dtest.o main.o libdtest.so main
dmd -c dtest.d -fPIC
dmd -oflibdtest.so dtest.o -shared -defaultlib=libphobos2.so -L-rpath=/usr/lib/

gcc -c main.c
gcc -rdynamic main.o -o main -ldl -lpthread


run ./build.sh, then run ./main

Interestingly, if you call test_fun directly from main() and not from a
detached thread, druntime doesn't throw asserts.

--