Which language futures make D overcompicated?

[Jonathan M Davis]

On Sunday, February 11, 2018 20:30:19 Nick Sabalausky  via Digitalmars-d 
wrote:
> The langauge-based package managers just simply need to keep
> "buildsystem" OUT of the package manager's scope. That's all. THAT is
> why system-level packages can be built with whatever tool: because the
> one thing the system package managers actually do get right is NOT
> rolling their own mandatory buildsystem.

I'm not sure that it's entirely a bad thing that dub includes a build system
in it, since it's the ease of use of its build system that is part of why
it's so worth using. It's just that its simplicity is also part of why it
doesn't work well once you get out of the basic situation of "here's a group
of D files that need to be compiled together as a library or application."

So, I do agree that in principle, we would be much better off if dub weren't
so tightly coupled with its build system. If dub were defined in a way that
allowed packages to use whatever build system they wanted so long as they
provided the build artifacts in a standard way to the package manager so
that they could be linked into other dub projects, we'd have something a lot
more flexible. And if done right, I think that it would be much better.
dub's default build system could still be used, but it wouldn't be forced.

The problem with that approach though (and part of why I think dub doesn't
do that) is that if dub really is set up with the idea that you use whatever
build system you want, then you have the issue of whether the system doing
the build has all of the necessary software to do the build. Dub doesn't
completely avoid that as-is, since you can depend on certain libraries being
on the system, and you can't write scripts to be run as part of the build
which could then require who-knows-what on the system (they're just
difficult to get to work with dub in a number of cases because of how much
dub assumes). But if dub itself were designed with the idea that you could
use cmake, scons, make, or whatever build system anyone felt like putting
together, then there's a fairly high chance when you go to pull in a package
from dub that it's going to require stuff to build that you don't have on
your system, and since you frequently end up pulling in packages recursively
such that you could be using dub packages that you've never even heard of,
it could become difficult to ensure that you have everything you need to
build everything in the dub package chain.

So, dub's solution of targeting a single build system side steps that issue
to a great extent - not entirely, but enough that it doesn't tend to be a
problem. And on some level, that's a good thing, but it comes at a definite
cost when you actually need something fancier.

So, I don't know what the right solution is. I'd like to see dub do better
interacting with arbitrary build systems, and if that can be done well, I
think that dub would be much better, but it's a hard problem. Ultimately
though, dub seems to have been designed to solve the common case while not
really making the general case very tractable (maybe not impossible, but
definitely difficult), but its place as the way the way to pull in 3rd party
libraries in D means that it arguably needs to be doing more than just solve
the common case, or 3rd party libraries that don't work with the common case
become intractable, which is definitely not good.

We would probably be better off if someone could come up with an alternate
package manager that was at least compatible enough with dub to use
libraries that use dub so that it could both work with dub and compete with
it and maybe ultimatelly get things right, but package and build management
is one of those hard problems that no one wants to work on, and plenty of
folks like to complain about.

As with too much around here, a big part of the issue is ultimately man
power. Even if we could all agree on exactly how D's build and package
management situation should be solved, actually get the work done is a huge
problem.

- Jonathan M Davis