DUB - call to arms

[H. S. Teoh]

On Wed, Apr 17, 2019 at 08:10:08PM +0000, JN via Digitalmars-d wrote:
[...]
> In the ideal world, perhaps. I am just worried opening dub to other
> build systems will just open a can of worms. Packages will use
> different build systems, which will be a pain to set up. Right now,
> the advice is "to get started with D, install official DMD
> distribution and set up your project with dub". Afterwards, the advice
> may be "to get started with D, install official DMD distribution, set
> up your project with dub, and install git,svn,meson,scons,make,cmake
> so that you can build some packages. Oh and set these environmental
> variables for these tools. Oh and if you're on Windows you're
> screwed".

Again, this concern is addressed by my proposal of build system wrapper
dub packages.  Basically, the act of declaring a dependency on, say, the
'make' wrapper package, should cause dub to download and install a
working installation of make.  This can be done by structuring the
'make' wrapper package such that its "build" command is to download and
install a working version of make.

In a nutshell, it would work like this:
- Project P depends on project Q, so dub downloads project Q.
- Project Q depends on 'make', so dub downloads the 'make' wrapper
  package.
- The 'make' wrapper package's "build" command is actually an installer
  that downloads and installs make on the local system. For maximum
  non-intrusion, the installation location can be somewhere in a
  standard location in the dub cache, so it won't conflict with any
  other make installation you may have on your system. So dub invokes
  this "build" command, and now you have a working version of make.
  (Note that this includes setting up any environment variables that may
  be necessary.)
- Now that project Q's dependencies are satisfied, it invokes Q's build
  command, which runs make to build everything. This now works, because
  the previous step ensures we have a working version of make installed.
- Now that project P's dependencies are satisfied, we can build P as
  usual.

This is one example of why abstracting away the "build" command of a
package from the package manager can be highly beneficial: it allows you
to pull in any arbitrary external package without needing dub to be
aware of how said external package works.

In the above scheme, you can substitute 'make' with any build system of
your choice, and it will still work with no additional effort from the
end user.  This is ensured by the wrapper package abstracting away the
dirty details of how to install a working version of ${buildtool} so
that dub can perform the installation without needing to know the
details of how to do it.

The wrapper package also hides away the dirty details of how to setup a
build tool like make properly, including setting up environment
variables and what-not.  Basically, it does whatever it takes for the
whole process to be completely transparent to the user.


> While dub may seem limiting to some, it offers standarization and
> unification. I know these are forbidden words in D community, but
> standards and forcing people to do things certain way are often
> beneficial and boost the cohesion of the ecosystem.

Standards should be empowering, rather than arbitrarily limiting.  What
we're proposing here is a standard way for arbitrary software packages
to be used as dub dependencies.  Let me repeat that again: what we're
proposing here is a *standard* way for *arbitrary* software packages (of
any language, any build system, etc.) to be usable as dub dependencies.

The standard here is the standard "API" of how packages work:

1) A package can depend on one or more packages, with optional version
constraints. This is the way dub already works.

2) A package can consist of any arbitrary collection of input files.
What they are and how they are structured is irrelevant.

3) A package has a standard declaration of how it should be built.
Basically, this is any arbitrary command that transforms said input
files into output files. How this is carried out is irrelevant -- that's
a detail the package is responsible for, and is none of the package
manager's business.

4) A package has a standard declaration of where to find the built
products (import paths, library files, etc.).

5) A package that wraps around a build system has a standard declaration
of how to formulate commands to invoke it for various standard tasks
(build debug, build release, test, etc.). So dependent packages don't
actually have to know how to run make on Windows vs. Linux, it uses a
standard query to formulate the right command, and simply runs it. The
wrapper package ensures that this command will Just Work(tm).

Just like the range API, the user of the range (the package manager)
doesn't, and shouldn't, care about how each of these "methods" work
under the hood.  As long as they conform to the API (declare items (1)
to (4) in a standard way), that's all that's necessary for dub to do the
Right Thing(tm).


> "I find this to be D's biggest current stumbling block, even moreso
> than anything in the language or compiler" - really? The biggest D
> stumbling block is the fact that it's not easy to set up a mixed C/C++
> project with D?  How many projects of that kind are there even?

Given the amount of effort currently being put into C++ interop?  I'm
guessing it's a lot more common than you think.


> How about issues such as crappy std.xml long overdue for replacement
> (I could list more but I don't want to derail this thread)?

Jonathan's dxml is already a dub package, isn't it?  You could just use
that.


T

-- 
If it tastes good, it's probably bad for you.