llvm-d

[Moritz Maxeiner]

On Sunday, 17 March 2013 at 20:05:09 UTC, Jens Mueller wrote:
> Deimos just provides a plain D interface for C libraries.
> What do you mean?
> You can do static or dynamic linking.
> With dynamic linking it will be loaded at startup time by the 
> loader.
> But you can also load it at runtime using some system API which 
> I assume
> you do. I started ddl (http://jkm.github.com/ddl/ddl.html) 
> which does
> work with Deimos to some extent. A limiting factor is 
> compilation speed.

Technically deimos provides extern(C) functions, which by 
themselves have - as  you said - to be linked 
statically/dynamically and cannot be loaded at runtime.
Normally to load at runtime you need a function pointer to give 
the OS functions to load from the shared lib and afaict you're 
using ctfe to extract the functions from deimos and create those 
functions pointers from them to circumvent that problem - kudos 
by the way, never even thought about that possibility.
Now to the real problem: When you're creating a shared lib on 
windows (32bit) it's going to be COFF, normally. But optlink 
can't link with that, only with OMF - which is one of the reason, 
I believe, why support for the MSLinker was added to windows 
64bit versions of dmd. You can use import libs to make dmd on 
win32 work with COFF shared libs, but again, that is extra work 
on the user of llvm-d, which I do not want. The solution for me 
was simply to cut out the middle man reponsible for that hassle 
(optlink) from the process.

On another note:
- Afaict ddl doesn't have OS X support, so I can't use it as a 
replacement for what I already have. If you want to add that to 
ddl you could just look at llvm/util/shlib.d in llvm-d, it's done 
there, missing support for one of the three majors is a big no-no 
for me.
- Another problem would be that I want to keep things compatible 
with dub, which would become very difficult seing as ddl doesn't 
have support for dub at present.
- Moreover, ddl doesn't seem to have support for loading from a 
specific directory (such as "lib"), or at least I can't find it
- There are functions in the LLVM C API, which are not available 
in every LLVM shared lib: The target specific ones. That means 
that - depending on which targets where selected when compiling 
th shared lib - these functions may or may not be in the shared 
lib and trying to load them from the shared lib MUST NOT throw an 
exception. If loading fails it simply means the shared lib 
doesn't have support for that specific target; ddl however, 
always throws an exception when it can't load a symbol.

Only with those issues resolved in ddl would it be even feasible 
for me to do a split into say llvm-d + (deimos)llvm and have 
llvm-d depend on ddl.
And even then llvm-d would depend on 2 projects (ddl and 
(deimos)llvm) instead of 0 like now and more dependencies for 
very little to no functionality increase is a very big argument 
against doing so for me.

> Even though I understand your arguments. Technically, they are 
> different
> projects. As you say built on LLVM's C API. One could use git 
> submodules
> to ease building the higher level API.

By that same argument, however, you could also claim that the D 
runtime core should be split into more projects - one who only 
imports the necessary C functions from the libc, and the other 
who does all the D stuff.
Not the best example, I know, but the point still stands: When 
the D OOP API is done, the llvm-d user should have no more need 
to use the C functions by themselves. Also LLVM A API functions 
often do strange things you wouldn't expect and the documentation 
is sometimes just simply wrong; using the LLVM C API from D with 
D's GC without something in between that takes care of the 
GC/NonGC gap is dangerous at best, I believe.

> In sum you want to load libraries at runtime and have a higher 
> level
> API. Both are fine. But I question that it is a good decision 
> to do the
> loading at runtime for each C library.

Not that I advocate doing it for every C library, but if one 
really wants to what would be the real downside? The program 
startup time would suffer, of course, but with current computer 
architectures would it really be by an amount that actually 
hurts? And would the comfort of not having to worry about 
different library formats and linkers  on different OS's outweigh 
that? It's not like it is actually calculation-heavy stuff, only 
loading symbols. You could even parallelise it if it's more than 
a couple of hundred.

> I would use Deimos bindings. I even started one myself.
> https://github.com/jkm/deimos-llvm/
> But I cannot devote much time on it.

That would make it one for and one somewhat against^^
In all seriousness, though: Even if I were to split llvm-d into 
the two parts, I can't do it right now, as only the C API stuff 
exists at present, so splitting would leave me with one empty 
github repository right now.
What I could do, however, is copying the C parts from llvm-d 
instead of splitting them. If I were to do that, I'd need to know 
how to get it in the deimos "D programming" group, though.

To sum it up: I'd much rather make a copy of the C stuff from 
llvm-d, make it a deimos project, but leave llvm-d untouched then 
to do a split and deal with the consequences. It might seem 
contraproductive, but the actual effort to support two very 
similar projects with updates is less than to deal with problems 
arising from the split.

- Moritz