Thoughts about D package management on Gentoo

[Marco Leise]

Am Sat, 09 Nov 2013 08:46:35 +0100
schrieb "Wyatt" <wyatt.epp at gmail.com>:

> I'm falling asleep here, but before I go...
> 
> On Saturday, 9 November 2013 at 00:38:16 UTC, Marco Leise wrote:
> >
> > "dub" which don't integrate with the official package
> > managers.
> >
> With the right functionality, we could do this similarly to how 
> we hook CPAN.pm for g-cpan.  If nothing else, it could be done a 
> damn sight more reliably than Diego's gem eclass if we get this 
> squared away sooner than later.  (This is a weird space, though.)

That sounds like an option. It might require some extensions
to dub though so we can have a library installed multiple
times for different compilers and D versions.

> > When I decide to install a D application on Gentoo, I want it
> > to just work. For Gentoo that includes the choice of compiler
> > to build it and the used compilation flags.
> >
> > The package manager should handle installing LLVM, the latest
> > stable LDC and required libraries on top of that and finally
> > the application.
> >
> For a long time, GDC was just a USE flag on GCC.  Not sure what 
> the current state is because I've been trying to ignore the sad 
> state of our D packaging in hopes that it would go away.  (How 
> many D overlays have we killed at this point anyway?)

There is a toolchain.eclass that creates the complete list of
IUSE for GCC, excluding 'd' for now. But without the front-end
source code in the source package it doesn't do anything
useful anyway.

> > - Compiler and library are closely coupled and depend on each
> >   other.
> 
> So we'll need a virtual for "the D implementation"?  cf. 
> virtual/package-manager and dozens of others.

Yes, I had virtual/phobos in mind, but you guys a probably
right that it should be more general like virtual/dlang.

> > - Each new D point release brings a new, generally
> >   incompatible standard library.
> 
> Unless I've misunderstood completely, the compiler, runtime, and 
> standard library are ostensibly separable, but for the 
> foreseeable future they need to be revbumped in lockstep for this 
> reason.  No biggie.  cf. kde-base/*

I wouldn't separate Phobos out just yet. One package per D
implementation will do the job.

> > - All three major compiler backends are in good shape and use.
> >   But their generated code is not exchangeable (as far as
> >   I've heard due to ABI differences).
> 
> Do they? I thought this had been worked out.

I asked Mr. Buclaw recently and he told me that is was still
an issue, at least when it comes to dmd.

> > - Tools tend to expect that "dmd" is available as a command.
> >
> Which tools?  Quantify the problem, and we may be able to fix it. 
> (We have like all of upstream right here...;)

Hehe, I can't really quantify it, but assumed that "dmd" is the
default for rdmd, dub, Mono-D and other tools. For many you
can override the compiler name, but it could still be helpful
to have /bin/dmd point to /bin/ldmd2-0.12.0 if LDC is the only
installed D compiler.

> > To deal with the complexity of package dependency management
> > in this scenario, we need to be able to install multiple
> > versions of D compilers in parallel (similar to what dvm does).
> 
> Do we really?  Not necessarily disagreeing, but I think this 
> needs substantiation.

Ok, fair enough... We have D compilers that excel at different
things. A big selling point is the rapid turn around time of
code changes when compiling with dmd, but its optimizer is
lacking on modern CPUs. My choice is to setup my debug builds
to use dmd and the release builds to use gdc/ldc. Also
sometimes one compiler has a corner case with some code and it
helps to try another one till a bug fix is available. (So much
for the value of multiple compilers, on to multiple
versions...)
Some projects are still written in D1. And with the Phobos
library as an .so now, we have to see how we can keep multiple
versions of Phobos in case some application isn't updated for
half a year and happens to need that last Phobos version.

> > This can be accomplished by so called "slots" for each point
> > release. We can do this for all three compilers and will end
> > up with binaries like "dmd2.064" and "gdc2.063".
> >
> See, I'd like to avoid the awful version creep we see with GCC-- 
> that only exists because we're trying to provide an upgrade path 
> for lazy people who never upgrade (never mind that it's never 
> really worked well).  Thankfully RAP may finally be fixing that, 
> now that Benda is joining the toolchain herd.

Some time you need to give me a who-is-who in Gentoo. Ok, just
what do you think is tolerable? gcc has quite a few versions,
but it doesn't really affect me. I have only 4.7.3-r1
installed. For D the story is different since the compiler and
standard library ship together and - from a packager's point of
view - it is all new every 2 to 5 months. If you have some D
programs installed that haven't been updated in a while you
have to keep old versions of phobos.so and other dependencies
around.

> > Next a symlink should be established to the current
> > implementation of dmd, e.g.: /bin/dmd -> /bin/ldmd2.
> > This symlink could be managed with an eselect plugin as it is
> > already done for other languages like Python or the Java
> > Runtime.
> >
> I'm so not fond of those, but it looks like we may not have a 
> choice.  Do we have any use for an env.d entry here?

I have never looked at those in depth. Possibly yes, since
there is Python and GCC already.

> > Since libraries for D depend not only on druntime/Phobos, but
> > also on the compiler the Gentoo user should be able to setup
> > globally and for each library which compilers it should work
> > with.
> 
> It's way too early for this shade of bikeshedding.  Let's see 
> about actually getting packages in the tree, first.  (The 
> approach may or may not be a good fit.  I think we should give it 
> some time to settle first, though.)

Hmm. This is actually my main motivation: Getting packages
like Derelict or GtkD working with both dmd for debug builds
and gdc for release builds without manual recompilation,
changing lib paths and library names, just by setting
D_COMPILERS="dmd gdc" in make.conf.

> -Wyatt

Thanks for your input. It is nice to talk about our options
here and to hear from few more D users on Gentoo that way.

-- 
Marco