Pyd on Linux building woes

[Kirk McDonald]

This is a lengthy one, for which I apologize. I've tried to lay the 
problem out as best I can, but I am not sure exactly where the problem 
lies. I am reasonably certain that it is a compilation or linkage 
problem, and not a problem with the code.

I have done some work with CeleriD and added support for GDC. This is 
required for Linux support, as DMD can't build shared objects on Linux. 
However, I've run into some basic building issues that I could use some 
help with.

Before I started mucking around with it, CeleriD simply took all of the 
code files needed, jammed them all on a command line with the 
appropriate options (-c, mostly), and compiled them. Then it linked them 
together. Fine.

When adding gdc support, the first thing I realised was that gdc (and 
gcc) doesn't support DMD's -od option, and distutils requires that all 
of the object files be placed in a specific directory. My first stab at 
getting around this was to compile the sources one at a time, specifying 
the location of the object files one by one.

When I tested this with DMD, it compiled but did not link. It complained 
of undefined symbols. The symbols given were the big long mangles of 
template instances. I am still not sure why this happened. I fixed it by 
putting it back the way it was, as a single call to the compiler.

To get rid of the need for the -od option, I changed the current working 
directory to where the object files needed to go, and specified the full 
path to each of the source files. By not specifying a location, the 
object files just get dumped in the cwd. Problem solved.

Except that gdc still doesn't work. The .so compiles and links nicely 
enough, except that when it is imported by Python, Python issues this error:

ImportError: /full/path/to/testdll.so: undefined symbol: 
_D3pyd9func_wrap56__T9func_wrapS25_D7testdll7dg_testFDFZvZvVk1TvTPFDFZvZvZ4funcUPS6python8PyObjectPS6python8PyObjectZPS6python8PyObject

Which is the mangle of a template instance. One of the very ones, I 
might add, that Optlink wasn't finding when I compiled the source files 
one by one.

For reference, the calls to the compiler and linker are:

DMD in Windows:
compiler:
 >C:\dmd\dmd\bin\dmd.exe
-c
-debug -unittest
-I<location of the Python header>
-I<location of Pyd>
<all of the source files: Python, Pyd, boilerplate, user code>

linker:
 >C:\dmd\dmd\bin\dmd.exe
-debug -unittest
-ofbuild\lib.win32-2.4\testdll.pyd
<the python DLL's .lib file>
<a list of the .obj files>
<the generated .def file>

That /works/. This doesn't:

GDC in Linux:
compiler:
$ gdc
-fPIC -c
-fdebug -funittest
-I <location of the Python header>
-I <location of Pyd>
<all of the source files: Python, Pyd, boilerplate, user code>

linker:
$ gdc
-fPIC -nostartfiles -shared
-fdebug -funittest
-o build/lib.linux-i686-2.4/testdll.so
<a list of the .o files>
-Wl,-soname,testdll.so
-lpython2.4

-nostartfiles is needed to override the _init and _fini functions, which 
are called when the module is loaded and unloaded (they are defined in 
the boilerplate code). -shared and the -soname option are similar to the 
.def file in Windows. (They tell the linker to make a .so file.) The 
final option is like the .lib file in Windows; it links it against Python.

It is possible that this is a problem with gdc? Or am I making some 
stupid mistake?

-- 
Kirk McDonald
Pyd: Wrapping Python with D
http://dsource.org/projects/pyd/wiki