Replacing Make for the DMD build

[Vladimir Panteleev via Digitalmars-d]

On Saturday, 17 June 2017 at 21:49:29 UTC, Walter Bright wrote:
> It's not quite the same. I spend 99.99% of my time programming 
> working with code, not makefiles.

Martin and Sebastian can correct me if I'm wrong, but unless I 
am, Martin, Sebastian, and myself spend a scary amount of time 
wrestling with makefiles. We have had bad production issues due 
to makefiles, such as missing dependencies causing inconsistent 
or broken builds (notably observable when something works only 
without -j), typos or undefined variables resulting in bugs being 
silently ignored (hey, I filed a PR to fix one of those not half 
an hour ago - dmd#6916), CI checks being accidentally completely 
switched off (which happened more than once!), the win32.mak / 
win64.mak / posix.mak mess, super-ugly hacks due to limitations 
of Make that slow down the build unconditionally and/or make 
everything much more fragile and complicated... and I could go on.

Which is not to say that I think we need to get off makefiles 
ASAP. Migrating to anything else is going to undoubtedly incur a 
massive migration cost and for a long time, a big maintenance 
cost (especially if it involves dogfooding). I agree with many of 
your arguments as well.

I think you're not encountering much of the problems with 
makefiles because you're mainly dealing with DMD, whose build 
process is relatively simple - and even there, the build process 
is mostly maintained these days by other people. In comparison, 
the build process for dlang.org has been getting much more 
complicated with time (partially because of some technical debt 
and fragmentation, but also simply because we want to do more 
things like nice runnable examples, various pre-processing / 
post-processing / validation, building the spec from the compiler 
source, etc.

If anyone wants to SERIOUSLY propose a plan to migrate from 
makefiles, I'd be interested to look at it. However, its benefits 
obviously must outweigh the maintenance costs of current 
makefiles, as well as not raising the contribution barrier too 
much.

Some requirements would be:

- Obvious syntax - making a change such as adding a new target or 
dependency should be obvious just from looking at the existing 
code.

- Performance - shouldn't have considerable overhead; but also, 
it shouldn't take 10 seconds to "rebuild" the build tool itself 
after every modification.

- Cross-platform support - this may seem obvious, but it should 
support at least all platforms DMD supports. One of the 
suggestions for replacement in a similar recent thread 
(http://forum.dlang.org/post/ohkkqj$21lg$1@digitalmars.com), 
buildsome, shows no indication of any Windows support at all, not 
even plans to work on it.

- Minimal dependencies - ideally it should work with as few as 
possible external dependencies that one would typically need to 
install to build D. Windows is very often overlooked when 
evaluating this requirement, for example although Python is 
ubiquitous on POSIX systems, it's much less so on Windows.

That doesn't leave many candidates. reggae might be the closest 
to satisfying all of the above, though I haven't looked too 
closely at it. For one thing, I don't really understand the need 
for build system generators - seems like an extra step and 
imposing an implementation detail on the user.