D build and SCons [was Answers needed from those using D for Web Development, Web APIs and Services]

[Russel Winder]

On Fri, 2017-12-22 at 10:39 -0800, H. S. Teoh via Digitalmars-d wrote:
> […]
> 
> For D projects, I've been finding that Command() has been the best
> tool
> for me in terms of configuring exactly how I want things built. I
> used
> to use (early versions of) your SCons D build tools (and thanks for
> that!), but ultimately went back to Command() because I found it very
> frustrating to have my builds break because of an incompatible change
> in
> the D tooling whenever I upgrade SCons.  So until the SCons D tooling
> API has stabilized, I'll probably hold off for the time being.

This is sad because without users the D support in SCons will not
improve.

There have been no changes in the D support SCons 2.6 → 3.0 other than
adding ProgramAllAtOnce so I have no idea what you found that broke. We
need a test case so as to fix it for 3.0.2 or 3.1.0

> Also, for vibe.d projects, I've been finding the need to write my own
> scanner in order to pick up Diet template (*.dt) dependencies, so
> that
> builds would trigger correctly when Diet templates are changed. There
> is
> no standard way to do this, unfortunately; so far I've been scanning
> for
> `render!(.*)` lines, but this doesn't always work if `render` is
> instantiated with parameters generated from CTFE.  Manual hardcoding
> has
> been necessary to get this part of my dependency tree to work.

Let's write a standard publish it via SCons_D_Experiments initially and
then put it is SCons Contrib or into the distribution.

People doing their own thing and not sharing is a good way of not
getting good things into the core.

> In the long term, I think an approach similar to tup will have to be
> adopted. O(n) dependency scanning just doesn't cut it anymore for
> code
> the size of today's large software projects. And with dynamic
> dependencies (e.g. CTFE-dependent imports) that are bound to happen
> in D
> code with heavy metaprogramming, there's really no sane way to manage
> dependencies explicitly; you really need to just instrument the
> compiler
> and record all input files it reads the way tup does.  I shouldn't be
> needing to write custom scanners just to accomodate CTFE-generated
> imports that may change again after a few more commits. It's SSOT
> (single source of truth) all over again: the compiler is the ultimate
> authority that determines which file depends on what, and having to
> repeat this information in your build script (or independently derive
> it
> via scanners) introduces fragility / incompleteness into your build
> system.

Again unless we do something nothing will change.

I am not sure you can get away from some element of O(n) behaviour if a
build system is to detect what is to be rebuilt in a compile then link
system. Obviously there are ways of minimising cf. Tup and Ninja vs.
Make and to some extent SCons. Tup still has a form of scan it is just
very fast due to the use of the file system tools it uses.

So if SCons is to be abandoned for D builds let's agree that and got on
with the tool that SCons and Dub are not.

[…]
> That's nice, though for now, I'm sticking with manually updating my
> dependencies when needed.  One thing I found annoying with dub was
> the
> sheer amount of time it spent at startup to scan all dependencies and
> packages and possibly downloading a whole bunch of stuff. The network
> latency really kills the compile-test-debug cycle time.  I know
> there's
> a switch to suppress this behaviour, but the initial dependency
> scanning
> is still pretty slow even in spite of that.  When a 1-line change
> requires waiting 15-20 seconds just to recompile, that really breaks
> my
> workflow.

I have been dithering with replacing the use of Dub itself, with a
SCons tool to replace Dub and work directly with the repository. Dub's
build structure really isn't useful for anything other than using Dub
as a built system.

Having two modes: update each time vs. only update when the developer
requires it is important. Unless a version glob is used, checking
dependencies should never take long.

> Plus, sometimes I *don't* want anything updated -- when debugging a
> program, the last thing I want is for dub or the build script or
> whatever to decide to link in a slightly different version of a
> library,
> and suddenly I'm no longer sure if the new crash is caused by the
> library or my own code, or the bug may now be masked by the slightly
> different behaviour of an upgraded library.

Isn't this consequent on the Dub version specification? If a specific
version is required this behaviour should not happen.

> I know that for people who want things done for them automatically
> and
> handed over on a silver platter, dub is great.  Unfortunately, it
> doesn't work for me. (But I also know that I don't represent typical
> usage, so take all this with a grain of salt.)

<panto-mode>
Oh no it isn't.
</panto-mode>

I am not a fan of Dub as a build system, but it appears to be the
accepted standard, or in my view sub-standard. (Trying to develop GtkD
code with Dub is a pain in the ####.)

Should the community push to ditch Make, CMake, SCons, Dub and use
Reggae (and hence Tup or Ninja)?

Not a simple question. For example CLion requires CMake. CMake-D
appears not to work so we can do D in CLion. Work on D in IntelliJ IDEA
is progressing but is relatively slow due to relying on volunteers.
Compare Rust which is now officially supported by JetBrains. This makes
a huge difference.

The develoment environment is almost as important as the programming
language.

-- 
Russel.
===========================================
Dr Russel Winder      t: +44 20 7585 2200
41 Buckmaster Road    m: +44 7770 465 077
London SW11 1EN, UK   w: www.russel.org.uk
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20171224/1feb912f/attachment.sig>