The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Monday, 10 April 2017 at 23:08:17 UTC, David Nadlinger wrote:
> On Monday, 10 April 2017 at 17:27:28 UTC, Matthias Klumpp wrote:
>> That's why I have been writing a lot of Makefiles and Meson 
>> build definitions lately.
>
> It seems like doing so without having a closer look at the 
> realities of D software (no stable ABI, etc.) might not have 
> been the best use of your time. How would Dub be such 
> fundamentally unfixable that making the official language tools 
> play nice with distro packaging (and the other way round) 
> wouldn't be preferable over manually re-writing build systems 
> all over the place?

Dub is not unfixable, but there are many more issues with it 
which make it very very hard to use in a distribution context - I 
tried that and it didn't work well.
I am treating D in Debian like I would treat C++, which is quite 
fair given the similar featuresets and challenges.

>> But with an unstable ABI the standard library is also 
>> affected, which would trigger us to do Haskell-style 
>> versioning (which mangles dependencies to depend on a virtual 
>> package containing a hash of the GHC version), and that not 
>> only sucks but also requires quite a lot of manpower.
>
> So it sounds like there is a solution already for other 
> languages. Could you elaborate some more on the problems with 
> it? I suppose there is some wiki page documenting the process 
> somewhere?

Haskell and OCaml permanently rebuild the whole stack on every 
new compiler release, which is why they have permanent transition 
trackers[1], so they basically continuously rebuild. I want to 
avoid this at all cost for D, as this is very very maintenance 
intensive and painful, and will require much more people to work 
on it than D has available in Debian.

[1]: https://release.debian.org/transitions/

>> Rust only has one compiler which strongly optimizes, so we 
>> don't have the problem of choosing the right one. Cargo is/was 
>> an issue but it's being worked on and seems to work well now: 
>> https://wiki.debian.org/Teams/RustPackaging/Cargo
>
> Rust doesn't have a stable ABI either, and it doesn't look like 
> there is any movement in that direction (not that I think that 
> there should be). That the people driving the effort might not 
> be aware of it yet doesn't mean it isn't an issue for them.

It's being worked on[2], but it's not a super-high priority.
There doesn't seem to be a definitive answer on how Rust is 
handled in Debian yet (but to know for certain, I would need to 
ask the Rust team).

[2]: https://github.com/rust-lang/rfcs/issues/600

> IIRC OCaml is also very much a statically linked affair. And 
> how does Debian distribute Go binaries? Is there any issue with 
> those being linked statically? If not, let's just distribute D 
> libraries as source and compile/link them statically when 
> building binaries, and problem solved.

Surprisingly it looks like many Go packages are indeed provided 
as source-installations.
Doing something like this with D would require Makefiles and Dub 
to pick up sources from system locations properly which isn't 
really done yet...

> Some of the compiler developers, myself included, understand 
> the issues involving ABI stability and distro packaging quite 
> well (although the latter admittedly only on a general level). 
> In fact, one of my earliest open source memories is of some 
> work in the trenches ensuring ABI stability of some bits of KDE 
> across releases. Yet we are still going to tell you that the D 
> ABI is going to remain unstable for the foreseeable future. 
> This is not something that just requires a man-week or month to 
> "fix" in the compiler, but would impact many other areas as 
> well, for example language evolution.
>
> If you somehow got the impression that this is just due to D 
> developers "not getting it", just have a look at the other 
> recent compiled languages. Go and Rust don't fare any 
> differently, and even Swift, with all its development manpower, 
> doesn't have a stable ABI yet [1]. And I believe header-only 
> C++ template libraries have been mentioned already as well.
>
> I can only speak for myself, of course, but I certainly see the 
> strategic importance of integration into the Linux distribution 
> ecosystem for D, and I'm very happy to work with packagers 
> wherever possible. However, you also need to acknowledge the 
> properties of the ecosystem *you* are working with. If you see 
> a big stretch of difficult terrain in front of you, closing 
> your eyes won't make it go away; you'll only lose time you 
> could spend working around it. ABI instability is something 
> you'll have to work around one way or the other.

That's the whole point of this thread, I want to find the best 
solution to deal with this issue and also one the D community can 
live with.
I am not set on any particular solution yet, at the moment I see 
the problem and I am thinking about how to deal with it in the 
best possible way (ideally not doing what OCaml/Haskell are 
doing, although pulling a "soft Haskell" where we rebuild 
everything when the D ABI changes would be way easier).