Dub, Cargo, Go, Gradle, Maven

[Jonathan Marler]

On Monday, 12 February 2018 at 10:35:06 UTC, Russel Winder wrote:
> In all the discussion of Dub to date, it hasn't been pointed 
> out that JVM building merged dependency management and build a 
> long time ago. Historically:
>
>   Make → Ant → Maven → Gradle
>
> and Gradle can handle C++ as well as JVM language builds.
>
> So the integration of package management and build as seen in 
> Go, Cargo, and Dub is not a group of outliers. Could it be then 
> that it is the right thing to do. After all package management 
> is a dependency management activity and build is a dependency 
> management activity, so why separate them, just have a single 
> ADG to describe the whole thing.
>
> SCons, CMake, and Meson (also Reggae?) are traditional build 
> tools, but they assume all package dependency management is 
> handled elsewhere, i.e. they state what is required for a build 
> but assume some other tool provides those packages, usually OS 
> package management, but note with C++ Conan is rapidly becoming 
> a big player. JFrog via Bintray and Artifactory do appear to be 
> the leaders in repository management and Gradle works bery well 
> with it.
>
> Rust, Ceylon, and D have to date chosen to eschew systems like 
> Bintray to create their own language specific versions for 
> whatever reason. This leads to language specific dependency 
> management and build. Go is also in this category really except 
> that Git, Mercurial, and Breezy (née Bazaar) repositories are 
> the only package storage used.
>
> Then, is a DevOps world, there is deployment, which is usually 
> a dependency management task. Is a totally new tool doing ADG 
> manipulation really needed for this?
>
> The lessons from all the tools from SCons to Gradle is that it 
> is all about ADG manipulation and constraint satisfaction. 
> SCons is really quite restricted even though it does it very 
> well (*).  Gradle really tries hard not just to solve the 
> problems of Maven (**), but to do end- to-end project 
> management well. In a sense it is the antithesis of each tool 
> does one thing and one thing only model, it is the "there is 
> one and only one ADG to describe the life of the project". 
> Maven and Gradle, and to a lesser extent Cargo and Go, 
> emphasise project management as a wholistic thing, rather than 
> making people deside on each item of the tool chain. Gradle 
> proves a good plugin system to allow changes to the default 
> standard project lifecycle.
>
> Gradle uses Groovy scripts or Kotlin scripts for project 
> specifications. Most projects are easily described by a purely 
> declarative specification using internal DSLs. However, for 
> those awkwards bits for some projects a bit of programming in 
> the specification solves the problem. So this goes against the 
> "a project specification must be purely declarative so use 
> TOML/JSON/SDL" but is easy DevOps worth it.
>
> Atila's Reggae has already shown how easy it is to use D (or 
> Python, Ruby, JavaScript, Lua) to define a build in an internal 
> DSL.
>
> Merging ideas from Dub, Gradle, and Reggae, into a project 
> management tool for D (with C) projects is relatively 
> straightforward of plan albeit really quite a complicated 
> project. Creating the core ADG processing is the first 
> requirement. It has to deal with external dependencies, project 
> build dependencies, and deployment dependencies. The initial 
> problem is creating an ADG from what is potentially a CDG and 
> then doing constraint satisfaction to generate actions. SCons 
> and Gradle have a lot to offer on this.
>
> Having been obsessed by build and project management since 
> about 1976, I'd be interested in doing some work on this.
>
>
>
> (*) The O(N) vs. O(n), SCons vs. Tup thing that T raised in 
> another
> thread is important, but actually it is an implementation thing 
> of how
> do you detect change, it isn't an algorithmic issue at a system 
> design
> level. But it is important.
>
> (**) Which many people ignore because Maven remains the major 
> project management tool in the JVM-verse.

Lot's of stuff here.  I would love for a build/package management 
tool to emerge and take over like git did with source control.  
However, my guess is that this problem is very hard and that's 
why there's so many tools that have their own pros and cons.  But 
it sounds like you're trying to sift through them all and find 
the gems in each tool.

I myself have made attempts to tackle the problem.  I created a 
tool called "dbuild" that allowed you to use D code to describe 
your project and then build it for you (kinda like Raggae I 
suppose).  And my latest project I call "bidmake" which is more 
general.  It implements the model of what I call "build 
contracts".  A project describes a "contract" and various 
"contractors" can be used to fulfill the contract.  You could 
have a "build contractor" that builds a contract, or a 
"documentation contractor" that generates documentation, etc.  
Anyway, I currently use the tool successfully on a few projects 
but I'm still not sure how much potential it has.  Maybe with 
your experience you can take a look at it and see if there's any 
substance to the idea. (https://github.com/marler8997/bidmake)