A brief survey of build tools, focused on D

Paul Backus snarwin at gmail.com
Sun Dec 16 00:17:55 UTC 2018


On Wednesday, 12 December 2018 at 22:41:50 UTC, H. S. Teoh wrote:
> It's time we came back to the essentials.  Current monolithic 
> build systems ought to be split into two parts:
>
> (1) Dependency detector / DAG generator.  Do whatever you need 
> to do here: dub-style scanning of .d imports, scan directories 
> for .d files, tup-style instrumenting of the compiler, type it 
> out yourself, whatever. The resulting DAG is stored in a 
> standard format in a standard location in the source tree.
>
> (2) Build executor: read in a standard DAG and employ a 
> standard topological walk to transform inputs into outputs.

You're missing (0) the package manager, which is probably the 
biggest advantage "monolothic" build tools like dub, cargo, and 
npm have compared to language-agnostic ones like make.

Granted, there's no reason dub couldn't function solely as a 
package manager and DAG generator, while leaving the actual build 
execution to some other tool.

> Every project should publish the DAG in a standard format in a 
> standard location.

You mean a Makefile? :^)

> Now of course, in real-life implementation, there will be many 
> more details that need to be taken care of.  But these are the 
> essentials: standard DAG representation, and a standard DAG 
> import function.

There's something important you're glossing over here, which is 
that, in the general case, there's no single obvious or natural 
way to compose two DAGs together.

For example: suppose project A's DAG has two "output" vertices 
(i.e., they have no outgoing edges), one corresponding to a 
"debug" build and one corresponding to a "release" build. Now 
suppose project B would like to depend on project A. For this to 
happen, our hypothetical DAG import function needs to add one or 
more edges that connect A's DAG to B's DAG. The question is, how 
many edges, and which vertices should these edges connect?

If we have out-of-band knowledge about A and B--for example, if 
we know they're both dub packages--then this is a relatively 
straightforward question to answer. (Though not completely 
trivial; see for example how dub handles the -unittest flag.) But 
if A and B can be absolutely *any* kind of project, written in 
any language, using any build tools, and able to produce any 
number of "outputs," there's no way to guarantee you've wired up 
the DAGs correctly short of doing it by hand.

One way to overcome this problem is to restrict projects to the 
subset of DAGs that have only one "output" vertex--but then, of 
course, you have to say goodbye to convenient debug builds, test 
builds, cross compiling, etc. So the cure may be worse than the 
disease.


More information about the Digitalmars-d-announce mailing list