A brief survey of build tools, focused on D

[H. S. Teoh]

On Tue, Dec 11, 2018 at 11:26:45AM +0100, Sönke Ludwig via Digitalmars-d-announce wrote:
[...]
> The upgrade check has been disabled in one of the latest releases, so
> unless the dependencies haven't been resolved before, it will not
> access the network anymore. A notable exception are single-file
> packages, which don't have a dub.selections.json - we should probably
> do something about this, too, at some point.
> 
> I've also rewritten the dependency resolution a while ago and it
> usually is not noticeable anymore nowadays.
> 
> Then there was an issue where LDC was invoked far too frequently to
> determine whether it outputs COFF files or not, making it look like
> scanning the file system for changes took unacceptably long. This has
> also been fixed.

This is very encouraging to hear.  Thanks!


> The main open point right now AFAICS is to make --parallel work with
> the multiple-files-at-once build modes for machines that have enough
> RAM. This is rather simple, but someone has to do it. But apart from
> that, I think that the current state is relatively fine form a
> performance point of view.

Wait, what does --parallel do if it doesn't compile multiple files at
once?


> > Then it requires a specific source layout, with incomplete /
> > non-existent configuration options for alternatives.  Which makes it
> > unusable for existing code bases.  Unacceptable.
> 
> You can define arbitrary import/source directories and list (or
> delist) source files individually if you want. There are restrictions
> on the naming of the output binary, though, is that what you mean?

Is this documented? I couldn't find any info on it the last time I
looked.

Also, you refer to "the output binary". Does that mean I cannot
generate multiple executables? 'cos that's a showstopper for me.


> > Worst of all, it does not support custom build actions, which is a
> > requirement for many of my projects.  It does not support polyglot
> > projects. It either does not support explicit control over exact
> > build commands, or any such support is so poorly documented it might
> > as well not exist.  This is not only unacceptable, it is a
> > show-stopper.
> 
> Do you mean modifying the compiler invocations that DUB generates or
> adding custom commands (aka pre/post build/generate commands)?

Does dub support the following scenario?

- There's a bunch of .java files that have to be compiled with javac.
   - But some of the .java files are generated by an external tool, that
     must be run first, before the .java files are compiled.
- There's a bunch of .d files in two directories.
   - The second directory contains .d files that need to be compiled
     into multiple executables, and they must be compiled with a local
     (i.e., non-cross) compiler.
   - Some of the resulting executables must be run first in order to
     generate a few .d files in the first directory (in addition to
     what's already there).
   - After the .d files are generated, the first directory needs to be
     compiled TWICE: once with a cross-compiler (LDC, targetting
     Arm/Android), once with the local D compiler. The first compilation
     must link with cross-compilation Android runtime libraries, and the
     second compilation must link with local X11 libraries.
      - (And obviously, the build products must be put in separate
        subdirectories to prevent stomping over each other.)
- After the .java and .d files are compiled, a series of tools must be
  invoked to generate an .apk file, which also includes a bunch of
  non-code files in resource subdirectories.  Then, another tool must be
  run to align and sign the .apk file.

And here's a critical requirement: any time a file is changed (it can be
a .java file, a .d file, or one of the resources that they depend on),
all affected build products must be correctly updated. This must be done
as efficiently as possible, because it's part of my code-compile-test
cycle, and if it requires more than a few seconds or recompiling the
entire codebase, it's a no-go.

If dub can handle this, then I'm suitably impressed, and retract most of
my criticisms against it. ;-)


T

-- 
Study gravitation, it's a field with a lot of potential.