Package manager - interacting with the compiler

[Jonathan M Davis]

On Saturday, December 10, 2011 22:17:44 jdrewsen wrote:
> On Saturday, 10 December 2011 at 08:55:57 UTC, Jacob Carlborg
> 
> wrote:
> > I think I've come so far in my development of a package manager
> > that it's time to think how it should interact with the
> > compiler.
> > 
> > Currently I see two use cases:
> > 
> > 1. When the package manager installs (and builds) a package
> > 
> > 2. When a user (developer) builds a project and want's to use
> > installed packages
> > 
> > In the best of worlds the user wouldn't have to do anything and
> > it just works. The package manger needs to somehow pass import
> > paths to the compiler and libraries to link with.
> > 
> > I'm not entirely sure what the best method to do this would be.
> > But I'm thinking that if the compiler could accept compiler
> > flags passed via environment variables use case 1 would be easy
> > to implement.
> > 
> > For use case 2 it would be a bit more problematic. In this use
> > case the user would need to somehow tell the package manager
> > that I want to use these packages, something like:
> > 
> > // project.obspec
> > orb "foo"
> > orb "bar"
> > 
> > $ orb use project.obspec
> > 
> > or for single packages
> > 
> > $ orb use foobar
> > $ dmd project.d
> > 
> > If environment variables are used in this case, then the
> > package manager would need a shell script wrapper, the same way
> > as DVM does it, to be able to set environment variables for the
> > parent (the shell). The reason for this is that a child process
> > (the package manager) can't set environment variables for the
> > parent process (the shell). This complicates the implementation
> > and installation of the package manager and requires different
> > implementations for Posix and Windows.
> > 
> > Another idea would be to manipulate the dmd.conf/sc.ini file
> > but that seems to be quite complicated and messy. On the other
> > hand, this wouldn't require any changes to the compiler.
> > 
> > Any other ideas?
> > 
> > https://github.com/jacob-carlborg/orbit/wiki/Orbit-Package-Manager-for-D
> > https://github.com/jacob-carlborg/orbit
> 
> For use case 1 the package manager could just as well call dmd
> directly with the correct flags ie. no need for using environment
> variables. Use case 2 does not belong to a package manager in my
> opinion. It is the job of a build tool to configure packages for
> a project.

This brings up an interesting situation. In general, I don't think that a 
package manager has any business building the project which is pulling in 
dependencies. However, it _does_ make some sense to build the dependencies on 
the box that these are being pulled in on, since they're going to have to be 
built for that box natively. And each of those projects could be using 
different build tools. One could be using make. Another could be using cmake. 
Another could be using scons. Etc.

So, how is that dealt with? Does each package list its choose build tool as a 
dependency and the programmer must then make sure that that build tool has 
been installed on their system by whatever means non-D packages/programs are 
installed? Or does that mean that packages using the package manager all need 
to use a specific build tool? And if they do, should the package manager then 
be that build tool? Or do we make it so that the package manager doesn't 
actually build _anything_? Rather it pulls in the source along with pre-built 
binaries for your architecture, and if you want to build it for your manchine 
specifically, you have to go and built it yourself after it gets pulled down?

This is all looking very messing to me.

I have no idea how orbit deals with any of this, since I've never really 
looked at orbit. But it makes for an ugly problem.

So, in general, I'd definitely prefer #1, but it may be that issues involved in 
this make #2 make more sense, but I'd have to study orbit in some detail to 
give a better opinion on it.

- Jonathan M Davis