Embedding D Shared Library in WSGI Web Server

[Chris via Digitalmars-d-learn]

On Tuesday, 28 October 2014 at 00:16:03 UTC, John McFarlane wrote:
> Hi,
>
> I've written a modest shared library in D that I'd like to call 
> directly from a Python web server (Linux/OS X, Apache, WSGI, 
> Pyramid). I can call it reliably from within Python unit tests 
> but on a running server, the library causes a SIGSEGV as soon 
> as I try anything as complicated as a writeln call. Under 
> Linux, I've tried using Pyd, CFFI and ctypes to bind the native 
> .so to Python with the same result. I've tried calling 
> Runtime.initialize();from an init function on server startup 
> and from the D function being called as part of the web 
> request. I've even tried writing a shim in C to make these 
> calls and call through to the D.
>
> I've compiled with DMD and GDC with a variety of switches. 
> Currently my build command looks like:
>
>     dmd ./d/*.d -version=library -release -shared -O -inline 
> -defaultlib=libphobos2.so -fPIC -oflibgusteau.so
>
> The extern(C) function takes a string and returns a string. 
> I've tried holding onto a pointer to the returned string and 
> passing in an adequately sized byte array from Python. Smaller 
> strings seem to cause a crash with lower probability but 
> typically the input and output strings are a few KB in size and 
> consistently crash. I taken measures to ensure that they are 
> correctly encoded and null terminated. I've also tried 
> disabling the GC, calling terminate on function exit (after 
> copying result into received buffer) and various other measures 
> in an attempt to get something working.
>
> I'm guessing that the web server spawns/relies on a thread or 
> process for each request and I've read that runtime 
> initialization should be invoked from the main thread. Does 
> this always mean the first thread that was created on process 
> start up or does it just have to be consistent? If calling from 
> a thread other than the one used to initialize, is that a 
> problem? Are other threads created when GC is invoked which 
> might last past the extern function being called and if so, can 
> I prevent this by controlling collection explicitly?
>
> Thanks, John

I had a similar problem with a DLL for Python. The reason my DLL 
would crash were some writeln statements. After removing them it 
worked fine. I suppose the writeln messes things up, be it 
because of threads or because of conflicts in the file system 
(remember writeln is a file system operation). I suggest you have 
your lib not print things to console, but use a different 
mechanism instead, e.g. pass a string back to Python and have 
Python do the printing (if needs be).

You could also try to set your lib up as a small socket server 
that waits for input and answers. That usually works for me.