Common makefile (gasp) stuff

[H. S. Teoh via Digitalmars-d]

On Wed, Jan 14, 2015 at 11:50:39AM -0800, Andrei Alexandrescu via Digitalmars-d wrote:
> On 1/14/15 9:01 AM, H. S. Teoh via Digitalmars-d wrote:
> >On Wed, Jan 14, 2015 at 04:27:10PM +0100, Jacob Carlborg via Digitalmars-d wrote:
> >>On 2015-01-14 00:20, Andrei Alexandrescu wrote:
> >[...]
> >>>I think it's time to reduce clutter and duplication by migrating
> >>>such common stuff into one common.mak file. Question is, where
> >>>should that file sit? One candidate is tools/ but it's not really a
> >>>tool. Another candidate is dmd/src/ because that would be the first
> >>>thing anyone depends on.
> >>
> >>Oh God, please no. I really, really hate that the makefiles are are
> >>looking in folders outside of the repositories.
> >
> >+1. Makefiles that reference paths outside the repo are really really
> >evil. They only ever work for the specific setup the author has, and
> >breaks for everybody else.
> 
> Currently there must be a place where druntime finds dmd for building
> purposes. Also phobos needs to find druntime. Also dlang.ord needs to
> find the source of both phobos and druntime.
> 
> There's no specific setup of one person as much as a boring:
> 
> /some/dir/
>   dmd
>   druntime
>   phobos
>   dlang.org
>   tools
> 
> I have a hard time making it any simpler.

I think you're missing the point. The problem is not that a specific
setup is required. The problem is that the user has to (1) know exactly
what this setup is, and (2) set it up *manually*. The first point is a
dire lack of documentation, the second point begs the question "why
require the user to do this when we can do this for them?". I.e., via
submodules, see below.

Not to mention, your proposed setup does not match dlang.org's
makefile's idea of what the setup should be. (For example, it generates
html files in dlang.org/web whereas the Phobos makefile generates html
files in phobos/../web, which appears to suggest that
{dmd,druntime,phobos} ought to be nested inside dlang.org instead of
being siblings as you have shown.) If even you can't keep things
consistent with what's in dlang.org's makefiles, what hope do the rest
of us have, let alone a casual user who's only tentatively interested in
D?


> >>>Ideas?
> >>
> >>If you really want to do something like this I would recommend
> >>creating a new repository, acting like a super repository, including
> >>the other repositories as submodules. Then you can create a new
> >>makefile that builds everything. Although, I still don't want the
> >>makefiles in the existing repositories be dependent on the new
> >>repository.
> >[...]
> >
> >+1, this is a much better approach than introducing yet more extra-repo
> >hard-coded paths into the makefiles. There are too many of those
> >already.
> 
> Where would that super-repo sit?
[...]

Huh?? What do you mean, where it would sit? If you set up a
"super-repo", it would look something like this:

	dlang/			# root git repo
		dmd/		# git submodule referencing the dmd repo
		druntime/	# git submodule referencing the druntime repo
		phobos/ 	# git submodule referencing the phobos repo
		dlang.org/	# git submodule referencing the dlang.org repo

Then checking out this new dlang repo (with the requisite git flags)
will create the above structure for you, including clone all the
sub-repos. *This* is what will address point (2) that I raised earlier:
the user shouldn't need to manually setup some arbitrary directory
structure for you -- automate this with git submodules instead!

Having such a super-repo will also solve the problem of git bisecting
earlier toolchain releases -- because the super-repo references the
sub-repos by SHA hashes, which are history-tracked by git, so we could
set it up so that each commit of the super-repo corresponds with a valid
version set of dmd/druntime/phobos. Then you could checkout a specific
commit of the super-repo and update git submodules, and you'll get a
snapshot of the entire toolchain as it stood at the time of the commit.
If we do this right, we could even have the autotester update the
super-repo automatically upon successful completion of the test suite
for the entire toolchain.

And ideally, stuff common to multiple subrepos should be in this
"super-repo", such as the stuff you're trying to factor out right now.
Including the master makefile that will build everything WITHOUT needing
to reference out-of-tree paths. And ideally, any out-of-tree references
needed by the subrepos will be via configurable parameters (e.g. $(DMD),
$(DRUNTIME_PATH), etc.), which the master makefile can set to the
correct values, so the subrepos don't have to know or hardcode where the
exact out-of-tree location ought to be.

Such a super-repo will also make release management easier: instead of
manually managing release tags in each of dmd/druntime/phobos, we can
tag the super-repo, which encapsulates the exact set of
dmd/druntime/phobos commits for that release.


T

-- 
There are 10 kinds of people in the world: those who can count in binary, and those who can't.