Dub, Cargo, Go, Gradle, Maven

[H. S. Teoh]

On Wed, Feb 21, 2018 at 11:03:13AM +0000, Russel Winder wrote:
[...]
> I have been (still am?) a SCons fan but it has some serious problems
> in some workflows.
> 
> I think all of the things you mention are solved in a system that has
> a general purpose programming language to describe the project and the
> build. There is clearly a tension between specifying a project in a
> purely declarative way (cf. Cargo, Dub, Maven) where all build and
> deploy activities are effectively hardwired (though Maven has plugins
> to amend things) versus systems that use a programming language.
[...]

I think the ideal situation straddles the divide between declarative
build specs and a full-fledged general programming language.  You don't
want it to get too general, lest you end up with the build equivalent of
spaghetti code where the build script becomes unreadable and
unmaintainable.  OTOH a purely declarative approach is limited by how
well the DSL is designed.  An insufficiently-expressive declarative
language leads to much frustration when you find yourself unable to
express something that you need to do with your build.

Ultimately, at the bottom level, we're just dealing with DAGs, so at
some level the build spec should simply be a bunch of node specs, where
each node consists of some set of inputs, some set of outputs, and a
computational black box (the build action) to get from the former to the
latter. This black box can be anything at all, and should not be
unnecessarily constrained.

On a higher level, though, specifying individual nodes may not always be
desirable (e.g., you want to be able to express things like "all .d
files in this directory", or "all recursive import dependencies of
source file F").  Typically you'd want a slightly higher-level language
for generating DAG node specs.  Here is where some serious thought needs
to be put into designing a language that's both straightforward for
common tasks, yet expressive enough to handle unusual tasks.  To abuse a
metaphor, DAG node specs are the assembly language of build systems, and
generally you want to "program" your builds in a higher-level language,
but you could either end up with the build equivalent of C with the
build analogues of unchecked array bounds, uncontrolled pointer
arithmetic, and hundreds of gotchas completely non-obvious to the
uninitiated, resulting in ugly, unmaintainable build scripts, or the
build equivalent of D with its expressive power yet packaged in a clean
syntax, resulting in clean build scripts that are comfortable to write
and maintain.

So far I've not yet decided which approach is better: using a
full-fledged programming language as the high-level API, like SCons
does, or a more constrained but more easily controlled (and
implemented!) declarative DSL that is not necessarily Turing-complete.
Either way, I prefer baking in as little special behaviour as possible,
in the sense that any special capabilities like build-in source code
dependency scanning or linker flag control ought to be built on top of
primitives that are exposed to user build scripts, so that at least in
theory, user build scripts could also attain to equivalent capability
without needing to hack the build tool.  Generally, I dislike any
disparity between what built-in constructs can do "magically" vs. what
user code (build scripts) can achieve, because that usually means that
things will work well as long as you remain within the parameters the
author has conceived, but should you ever encounter a situation the
author didn't anticipate, you're stuck up the creek without a paddle.


T

-- 
Philosophy: how to make a career out of daydreaming.