building shared library from D code to import into cython

[Laeeth Isharc via Digitalmars-d-learn]

Hi.

Thanks for the quick response.

The -defaultlib was left around from trying all kinds of 
combinations of dmd and gcc.  I am not used to gcc, and it will 
take me some time to become properly acquainted with all the 
options.

I simply could not get it to recognize libphobos no matter what 
path settings I tried passing.

http://forum.dlang.org/thread/k3vfm9$1tq$1@digitalmars.com?page=4

I don't know whether this was necessary, but I did manage to get 
it to work after looking at a thread from a couple of years back. 
  I recompiled the D runtime and Phobos with position independent 
code (PIC) and pointed everything to that version instead, and it 
worked fine.  (Just export PIC=True, rename the posix.mak to 
Makefile, change the DMD path to /usr/sbin/dmd - in my case, and 
it all worked).

Next step is to try calling D interface from Cython.


Laeeth

In case anyone else should struggle with this in future.  
Makefile here:

PHOBOS_PATH=/opt/dmd2/src/phobos/generated/linux/release/64


pytestpy.so: pytest.d pytestpy.o
	dmd -fPIC -c pytest.d  #-shared -defaultlib=libphobos2.so 
-L-rpath=.:${PHOBOS_PATH}
	gcc -shared pytest.o pytestpy.o -o pytest.so -lphobos2 -lpthread 
-lrt -L. -L${PHOBOS_PATH} -Wl,-rpath=.:${PHOBOS_PATH} -o 
pytestpy.so -defaultlib=libphobos2.so -L-rpath=.:${PHOBOS_PATH}

pytest.o: %.d
	dmd -c $<

pytestpy.o: pytestpy.pyx
	cython pytestpy.pyx
	gcc -fPIC -c pytestpy.c -o pytestpy.o -I/usr/include/python2.7


clean:
	rm -f pytestpy.c pytest.o pytestpy.o pytest.so


pytest.d

extern (C) long pytest(long a)
{
	return a*2;
}

pytestpy.pyx
cdef extern long pytest(long a)

cpdef pytestpy():
	return pytest(109)

print pytest(102)


setup.py
from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize

setup(
		ext_modules = cythonize([
  				 Extension("pytestpy", ["pytestpy.pyx"],
				libraries=["pytest"],
				)
  	]))



On Tuesday, 7 October 2014 at 22:46:09 UTC, Freddy wrote:
> On Tuesday, 7 October 2014 at 20:55:59 UTC, Laeeth Isharc wrote:
>> Hi.
>>
>> I am trying to create a shared library in D linked against 
>> phobos so that I may use this in a cython extension module for 
>> Python.  Ultimately I would like to be able to use a D class 
>> or struct (via the C++ interface) and call it from within 
>> cython, since cython classes cannot be instantiated without 
>> the gil (and this prevents easy parallelisation).
>>
>> I feel a bit foolish asking the question as there is a nice 
>> example here for working with plain C using dmd as the linker, 
>> and using dmd and gcc to create a DMD shared library 
>> statically linked to phobos.  However, I have not succeeded in 
>> creating a D library statically linked to phobos that works 
>> with cython and python,
>> http://dlang.org/dll-linux.html#dso7
>>
>> I tried it first with test C code to make sure I am able to 
>> get the C library/cython/Python interaction working.
>>
>> pytest.c:
>> #include <stdio.h>
>>
>> long pytest(long a)
>> {
>> 	return a+1;
>> }
>>
>> int main()
>> {
>> 	long a =pytest(100);
>> 	printf("%ld",a);
>> 	return 0;
>> }
>>
>>
>> pytestpy.pyx:
>> cdef extern long pytest(long a)
>>
>> cpdef pytestpy():
>> 	return pytest(109)
>>
>>
>> setup.py:
>>
>> from distutils.core import setup
>> from distutils.extension import Extension
>> from Cython.Build import cythonize
>>
>> setup(
>> 		ext_modules = cythonize([
>> 				 Extension("pytestpy", ["pytestpy.pyx"],
>> 				libraries=["pytest"],
>> 				)
>> 	]))
>>
>>
>> command line:
>> gcc -shared -o libpytest.so pytest.o
>> python setup.py build_ext -i
>> <copied libpytest.so to /usr/local/lib>
>> python
>> import pytestpy
>> pytestpy.pytestpy()
>> <it works>
>>
>> ----
>> now try pytest.d
>> import std.stdio;
>>
>> extern (C) long pytest(long a)
>> {
>> 	return a*2;
>> }
>>
>> void main()
>> {
>> 	auto a =pytest(100);
>> 	writefln("%d",a);
>> }
>>
>> command line:
>> rm pytestd.o
>> rm libpytest.so
>> rm /usr/local/lib/libpytest.so
>> dmd -c pytest.d -fPIC
>> gcc -shared -o libpytest.so pytest.o -defaultlib=libphobos2.so 
>> -L-rpath=/usr/local/lib
>> python
>>
>>>>> import pytestpy
>> Traceback (most recent call last):
>>  File "<stdin>", line 1, in <module>
>> ImportError: /usr/local/lib/libpytest.so: undefined symbol: 
>> _D3std5stdio12__ModuleInfoZ
>>
>>
>> I guess it is not linking to the D runtime, but I am not sure 
>> what I should be doing to fix.
>>
>> Any thoughts appreciated.
>>
>> (The next step I was going to try when this works was C++ 
>> interface vs importing as a Cython class, but I thought best 
>> to start simple).
>>
>> I am running this on 64 bit Fedora 20.
>>
>> Thanks.
>>
>>
>> Laeeth.
> Since when does gcc have a -defaultlib option?
> ---
> $ man gcc | grep  defaultlib
> object-file-name  -llibrary -nostartfiles  -nodefaultlibs
>              This is useful when you use -nostdlib or
> -nodefaultlibs but you do
>              -nodefaultlibs is used.
>          -nodefaultlibs
>              -nodefaultlibs is libgcc.a, a library of internal
> subroutines which
>              -nodefaultlibs you should usually specify -lgcc as
> well.  This
> ---
> and why do you have a main function when compiling a shared
> library?
>
> Anyway, I think the problem is that "libpytest.so" can't find
> "libphobos2.so". Try adding "libphobos2.so"'s location
> (/usr/lib/x86_64-linux-gnu/libphobos2.so on my linux mint) to
> your -rpath .