The D ecosystem in Debian with free-as-in-freedom DMD

Mon Apr 10 11:12:26 PDT 2017

On Monday, 10 April 2017 at 17:29:04 UTC, H. S. Teoh wrote:
> On Mon, Apr 10, 2017 at 11:40:12AM +0000, Matthias Klumpp via 
> Digitalmars-d wrote: [...]
>> [...]
>> If we do that, we will run into the D ABI trap: Libraries 
>> compiled with compiler X can not be used from software 
>> compiled with D compiler Y. There is actually no ABI stability 
>> guarantee even between DMD releases.  This will make 
>> integrating D a huge pain. Recompiling the dependency-chain of 
>> a software from source when compiling a package using the 
>> "right" compiler and statically adding the code is forbidden 
>> by distro policy.  Having static libraries in the dependencies 
>> doesn't solve the issue.  Compiling each library with all D 
>> compilers is highly impractical and not really feasible.
>
> This is not a hard problem to solve, IMO.  Just build the 
> library into two separate binaries, each with a SONAME that 
> encodes the ABI it is compatible with.

This would require to hack each and every build system to support 
this *and* if there is a pkgconfig file for the shared library, 
to change all depending software to check for multiple library 
names which is a bit crazy...

> The resulting two .so's can either be distributed as separate 
> packages (for minimum bloat, if that's a concern), or as a 
> single package that contains both binaries (since they have 
> different SONAMEs this should not be a problem).

Since one library == one package in Debian, it would have to be 
multiple packages, otherwise we would need to override Lintian 
errors/warnings, which is always a bad idea.

> Then if you compile some software X that depends on this 
> library, it will pick up the correct version of the library 
> depending on which compiler you compiled with.

Unfortunately not without special support in the software's build 
system :-/

> [...]
> DMD is unlikely to support other archs than amd64/ia32 in the 
> foreseeable future, so the justification for dmd being 
> unavailable for arch X would be that upstream DMD simply 
> doesn't support it.  This, however, should not prevent us from 
> using gdc/ldc on those other archs, so that we can still ship 
> packages for those archs.  They will merely require ldc rather 
> than dmd. And obviously, libraries built for that arch will 
> only support the ldc SONAME, not the dmd one. (This may be an 
> argument for bundling both SONAMEs in a single package -- it 
> gets messy if we start shipping multiple libraries, some of 
> which may not be available on all archs. By shipping a single 
> package that includes both versions for ia32/amd64, we can 
> simply omit the DMD-compiled version from other archs.)

Conditional build-dependencies are a bit annoying, but with a 
metapackage "d-compiler" or similar, using different D compilers 
on different architectures would definitely be possible.

> Unfortunately, I realize that this means some packages that 
> require the latest DMD would not be available on all archs, if 
> they require features unavailable in gdc/ldc.  But this problem 
> may not be a huge one, now that ldc is mostly up-to-date with 
> dmd (at most 1 release behind, IIRC). GDC may lag behind a bit 
> more because it is unfortunately tied to GCC releases, so we 
> may have to forego using gdc for building newer D packages. But 
> we should be able to ship most D packages compiled with both.

Compiling with multiple compilers is a really big effort with 
rather questionable gain, IMO.
But as far as LDCs compatibility with other D projects goes: That 
is really good, the only reason you sometimes can't compile some 
random D code with LDC might be bugs, but not old standard 
libraries.

> Furthermore, I wonder if we should standardize on ldc for most 
> D software instead of dmd, unless that software absolutely 
> depends on features only available on dmd.  My reasons are:
>
> - While dmd compiles very fast, it consistently (IME) produces 
> code that
>   runs about 20-30% slower than code produced by gdc (and I 
> presume ldc
>   as well). Since we're talking about building Debian packages 
> on
>   Debian's buildd's, which are background batch processes, 
> compilation
>   speed is no big deal, but the performance of the executable 
> *is* a big
>   deal. The last thing we want is to give a poor impression of 
> D by
>   shipping official Debian packages that have subpar 
> performance.
>
> - DMD is unlikely to target other archs than ia32/amd64 in the
>   foreseeable future, AFAIK, unless the recent relicensing 
> triumph of
>   dmd's backend makes this future more likely.  There will 
> definitely be
>   resistance among DDs because of lack of support for other 
> archs. LDC,
>   in contrast, supports all of Debian's supported archs.
>
> - LDC is already available in Debian, meaning that we can 
> already start
>   adding D packages built with ldc without needing to work 
> through the
>   red tape involved in adding a new compiler to the archive.

I agree with all of that, I think sticking with LDC might indeed 
be the least painful thing to do...

> - The only case where I can see we'd want to compile with dmd 
> is if the
>   target software uses features that are only available on dmd.

Oddly, it usually is the other way round :P - some projects 
explicitly want LDC or GDC.

> [...]
> Of course, this in no way diminishes the priority in 
> *distributing* dmd in the Debian archive as -dev packages. Just 
> don't build other Debian packages with it. :-D

Indeed. It needs license review (urgh... ^^), but when it's 
through that there is no reason not to have it in Debian main and 
Ubuntu.

> [...]
>> Also: If you want to help out Debian's D team, feel free to 
>> ping me or any other D team member (we are very short handed 
>> and are only two active people right now). See 
>> https://wiki.debian.org/D (caution, wiki page is very 
>> outdated, last touched in 2012)
>
> I'd like to be involved, since I *am* technically a Debian 
> developer (though haven't been very active for the past long 
> while -- I do try to at least keep the meager few packages I 
> maintain up-to-date, though).

It's unlikely that I will maintain the dmd package (I might 
co-maintain it or sponsor uploads though - reason is simply lack 
of time and -ETOOMANYPROJECTS), so any help there is highly 
appreciated and I'd help with it whenever I can (I just can't 
make long-term commitments).
If DMD is uploaded as part of the D team, spreading the 
maintenance cost will be rather simple too :-)