A brief survey of build tools, focused on D

Jonathan M Davis newsgroup.d at jmdavisprog.com
Wed Dec 12 21:52:09 UTC 2018


On Wednesday, December 12, 2018 1:33:39 PM MST H. S. Teoh via Digitalmars-d-
announce wrote:
> On Wed, Dec 12, 2018 at 10:38:55AM +0100, Sönke Ludwig via Digitalmars-d-
announce wrote:
> > Am 11.12.2018 um 20:46 schrieb H. S. Teoh:
> > > Does dub support the following scenario?
>
> [...]
>
> > This will currently realistically require invoking an external tool
> > such as make through a pre/post-build command (although it may
> > actually be possible to hack this together using sub packages, build
> > commands, and string import paths for the file dependencies). Most
> > notably, there is a directive missing to specify arbitrary files as
> > build dependencies.
>
> I see.  I think this is a basic limitation of dub's design -- it assumes
> a certain (common) compilation model of sources to (single) executable,
> and everything else is only expressible in terms of larger abstractions
> like subpackages.  It doesn't really match the way I work, which I guess
> explains my continuing frustration with using it.  I think of my build
> processes as a general graph of arbitrary input files being converted by
> arbitrary operations (not just compilation) into arbitrary output files.
> When I'm unable to express this in a simple way in my build spec, or
> when I'm forced to use tedious workarounds to express what in my mind
> ought to be something very simple, it distracts me from my focusing on
> my problem domain, and results in a lot of lost time/energy and
> frustration.

What you're describing sounds like it would require a lot of extra machinery
in comparison to how dub is designed to work. dub solves the typical use
case of building a single executable or library (which is what the vast
majority of projects do), and it removes the need to specify much of
anything to make that work, making it fantastic for the typical use case but
causing problems for any use cases that have more complicated needs. I
really don't see how doing much of anything other than building a single
executable or library from a dub project is going to result in anything
other than frustration from the tool even if you can make it work. By the
very nature of what you'd be trying to do, you'd be constantly trying to
work around how dub is designed to work. dub can do more thanks to
subprojects and some of the extra facilities it has for running stuff before
or after the build, but all of that sort of stuff has to work around dub's
core design, making it generally awkward to use, whereas to do something
more complex, at some point, what you really want is basically a build
script (albeit maybe with some extra facilities to properly detect whether
certain phases of the build can be skipped).

I would think that to be fully flexible, dub would need to abstract things a
bit more, maybe effectively using a plugin system for builds so that it's
possible to have a dub project that uses dub for pulling in dependencies but
which can use whatever build system works best for your project (with the
current dub build system being the default). But of course, even if that is
made to work well, it then introduces the problem of random dub projects
then needing 3rd party build systems that you may or may not have (which is
one of the things that dub's current build system mostly avoids).

On some level, dub is able to do as well as it does precisely because it's
able to assume a bunch of stuff about D projects which is true the vast
majority of the time, and the more it allows projects that don't work that
way, the worse dub is going to work as a general tool, because it
increasingly opens up problems with regards to whether you have the right
tools or environment to build a particular project when using it as a
dependency. However, if we don't figure out how to make it more flexible,
then certain classes of projects really aren't going to work well with dub.
That's less of a problem if the project is not for a library (and thus does
not need to be a dub package so that other packages can pull it in as a
dependency) and if dub provides a good way to just make libraries available
as dependencies rather than requiring the the ultimate target be built with
dub, but even then, it doesn't solve the problem when the target _is_ a
library (e.g. what if it were for wrapping a C or C++ library and needed to
do a bunch of extra code steps for code generation and needed multiple build
steps).

So, I don't know. Ultimately, what this seems to come down to is that all of
the stuff that dub does to make things simple for the common case make it
terrible for complex cases, but making it work well for complex cases would
almost certainly make it _far_ worse for the common case. So, I don't know
that we really want to be drastically changing how dub works, but I do think
that we need to make it so that more is possible with it (even if it's more
painful, because it's doing something that goes against the typical use
case).

The most obvious thing that I can think of is to make it work better to use
dub to pull in libraries without actually using dub as the build tool for
your project. That doesn't solve using dub to actually build your project
when it needs a complex build, but it makes working with it better when it
really doesn't make sense to use dub for more than pulling in dependencies.
Beyond that, I suspect that if we really wanted to make dub truly flexible,
we'd have to look into making it more plugin-based to allow alternate build
systems, but that's a _much_ larger shift in how it works. Regardless, it
would require manpower that isn't currently being targeted at dub.

- Jonathan M Davis






More information about the Digitalmars-d-announce mailing list