Makefile experts, unite!

[H. S. Teoh via Digitalmars-d]

On Mon, Jun 12, 2017 at 09:53:42PM +0300, ketmar via Digitalmars-d wrote:
> H. S. Teoh wrote:
> 
> > Don't forget tup, and others inspired by it, which use modern OS
> > features to reduce the cost of determining what to build to an O(1)
> > database lookup rather than an O(n) whole-source tree scan.
> 
> added complexity for nothing. disks gradually replacing with ssd,
> amount of RAM allows to cache alot, and CPUs are faster and faster
> (and has more cores).

CPU speed has nothing to do with this, disk roundtrip is always going to
be the bottleneck, even with SSD.


> i still have HDD, 8GB of RAM, and 32-bit system. and it even worse
> some time ago. in the last ~8 years i was using my k8jam for various
> projects: several kb to multimegabytes of code and thousand files/alot
> of subdirs. source tree scanning and dependency resolving NEVER was
> any significant factor.

The problem is not building the entire source tree -- that's not the use
case we're targeting.  We're talking about incremental builds, which are
not relevant to make because in make they are inherently unreliable.

Several KB is kids' play.  I work with a source tree with at least 800MB
of source code (not counting auxiliary scripts and other resources that
must be compiled as part of the build process), producing a 130MB
archive of *compressed* executables, libraries, and other resources.
The only way I can even maintain my sanity is to `cd
deeply/nested/subdir; make` to compile the specific target I'm working
on. Scanning dependencies take a *long* time, and can mean the
difference between being able to test a 1 line change in 1 second vs. 5
*minutes* while the thing scans 800MB worth of source code just to
recompile 2 source files.

It's either that, or scan the timestamps instead of the file contents,
like make does, which will be fast, but you then have the lovely
side-effect of unreliable builds and non-existent bugs that appear in
the executable but has no representation in the source code.  *Then* you
have to ditch incremental buids altogether and do wholesale 'make clean;
make', so it's time to take a nap for 20 mins while it recompiles the
entire source tree just to account for a 1-line bug fix.


> and i really mean it: it was less than a second even for a huge
> projects, where linking alone took long enough that i could get coffee
> and cigarette.  ;-)
[...]

Your project is not huge enough. :-D


T

-- 
Do not reason with the unreasonable; you lose by definition.