DIP11: Automatic downloading of libraries

[Adam D. Ruppe]

Nick Sabalausky wrote:
> You import a module *from* a library.

I don't think libraries, aside from individual modules, should even
exist in D, since you can and should put all interdependent stuff
in a single file.

If something is moved to a separate module, that means it has
usefulness independent from original module - if not, it wouldn't
be factored out to begin with...


A module might import a whole web of modules, but each one it
imports would still satisfy the definition of module - something
that's useful to more than one other modules.


Suppose I write a database "library". It is composed of three
modules: database.d which is the common interface, and then
mysql.d and sqlite.d that implement that interface for their
respective underlying engines.

The only shared component is that database.d interface. Does that
warrant making a library package for them?

I don't think so. Suppose a third party wants to implement access
to Microsoft SQL via my same interface.


If modules are the building blocks, that's easy for him. He just
"import adams.database;" and puts the file up. He doesn't have
to petition his module to be adopted by my database library while
still being compatible with it.

A user can then pull any one of the implementation modules:
adams.mysql or adams.sqlite or other_fellows.mssql and it just
works.



You could do the same thing with a library concept, but would you?
Do you download a whole library just so you can implement a shared
interface that is otherwise unrelated?


> Also, if a library needs any special "setup" step, then this
> won't even work anyway.

This is true, but I see it as a strike *against* packaged libraries,
not for it.

Going back to the database interface. Suppose I only offered
database.d as part of a "Adam's Database Library" package, which,
since it offers mysql, lists mysql as a dependency.

Then Microsoft implements my interface.


Someone who wants to use Microsoft's library is told it depends
on mine... which depends on mysql.

So it prompts them to install mysql to use mssql! That's awful.


To fix this, you might say "library mysql depends on library
database".... but, taking that to it's logical conclusion,
library == module anyway.



BTW, you might be thinking, if you import my mysql module, how
does it handle the C library it depends on? Answer: it doesn't.
That's the library user's responsibility.

If he's on CentOS, he can yum install mysql. If he's on Debian,
he can apt-get install mysql.


A D package manager shouldn't step on the toes of existing package
managers. Lord knows they have enough problems of their own without
dealing with our interop.


> I think a substantial number of people (*especially* windows
> users - it's unrealistic to expect windows users to use anything
> like junctions) would expect to be able to use an already-
> installed library without special setup
> for every single project that uses it.

The download program could automatically make locally available
libs just work without hitting the network too.

> A central repo per se, isn't really a good idea.

Agreed. I just like having the option there for max convenience in
getting started. The central repo might just provide a list of
other repos to try.

> I think things like apt-get and 0install are very good models for
> us to follow

Blargh. I often think I'm the last person people should listen
to when it comes to package management because the topic always
brings three words to my mind: "shitload of fuck".

I've never seen one that I actually like. I've seen only two
that I don't hate with the burning passion of 1,000 suns, and
both of them are pretty minimal (Slackware's old tgz system and
my build.d. Note: they both suck, just not as much as the
alternatives)

On the other hand, this is exactly why I jump in these threads.
There's some small part of me that thinks maybe, just maybe,
we can be the first to create a system that's not a steaming pile
of putrid dogmeat.


Some specific things I hate about the ones I've used:

1) What if I want a version that isn't in the repos? Installing
a piece of software myself almost *always* breaks something since
the package manager is too stupid to even realize there's a
potential conflict and just does its own thing.

This was one of biggest problems with Ruby gems when I was forced
to use it a few years back and it comes up virtually every time
I have to use yum.

This is why I really like it only downloading a module if it's
missing. If I put the module in myself, it's knows to not bother
with it - the compile succeeds, so there's no need to invoke the
downloader at all.


2) What if I want to keep an old version for one app, but have
the new version for another? This is one reason why my program
default to local subdirectories - so there'd be no risk of stepping
on other apps at all.


3) Can I run it as non-root? CPAN seemed almost decent
to me until I had to use it on a client's shared host server. It
failed miserably. (this was 2006, like with gems, maybe they
fixed it since then.)

If it insists on installing operating system files as a dependency
to my module, it's evil.


4) Is it going to suddenly stop working if I leave it for a few
months? It's extremely annoying to me when every command just
complains about 404 (just run yum update! if it's so easy, why
doesn't the stupid thing do it itself?).

This is one reason why I really want an immutable repository.
Append to it if you want, but don't invalidate my slices plz.



Another one of my big problems with Ruby gems was that it was
extremely painful to install on other operating systems. At the
time, installing it on FreeBSD and Solaris wasted way too much
of my time.

A good package manager should be OS agnostic in installation,
use, and implementation. It's job is to fetch me some D stuff
to use. Leave the operating system related stuff to me. I will
not give it root under any circumstances - a compiler and build
tool has no legitimate requirement for it.

(btw if it needs root because some user wanted a system wide thing,
that's ok. Just never *require* it.)