Orbit - Package Manager - Specification/ideas

[Nick Sabalausky]

"Jacob Carlborg" <doob at me.com> wrote in message 
news:j01dlr$nm$1 at digitalmars.com...
> On 2011-07-18 00:26, Nick Sabalausky wrote:
>> "Jacob Carlborg"<doob at me.com>  wrote in message
>> But why should the name of the project and the name of the tool be
>> different? It would be less confusing to have a packager manager named 
>> Orb
>> that gets invoked via "orb". (Or a package manager named Orbit that gets
>> invoked via "orbit".
>
> I don't know. It's also easy to separate the library code in one package 
> named "orbit" and the application code in on package named "orb".
>

What about "orb"/"orblib", or "orbit"/"liborbit", etc? Be easier to tell 
which is which that way anyway.


>> Also, I don't see why the name of "orbspec" needs to be prefixed with the
>> package's name. Just like an orbfile, or makefile, or rakefile, etc., you
>> already know what package it's for from what directory/package it's 
>> inside
>> of.
>
> The package needs a name. One option is to specify that in the orbspec 
> file. Another option is to specify it in the name of the orbspec file. I 
> just thought that it was convenient.
>

Ok, I see. That makes sense.


> Is it a good idea to take the name directory where the orbspec is located?
>

No, I don't think it is. When orb downloads/installs a package it should 
definitely put it in a directory named the same as the package. But I often 
have reason to projects in a directory with a (slightly) different name from 
the project, so I don't think they should be expected to be the same names a 
packages the user is developing.

So I'm fine with the orbspec filename being prepended with the package 
name...Only one minor potential issue I see: What happens if a single 
project has both "foo.orbspec" and "bar.orbspec"?


> Note that many of my choices are based on RubyGems.
>

I guess I should look at that :P


>> That would make sense. But, it could always be forcefully done by doing 
>> it
>> manually without going through orb. Don't know if that's worth worrying
>> about though.
>
> Do you mean if you have direct access to the server where the repository 
> is located?

Right.


> I don't think that's something worth worrying about.

Fair enough. I guess just as long as it's understood that that shouldn't be 
done because if it is done, other users won't get the new version "a.b.c" 
unless they uninstall and reinstall it.


>> I think it should be up to the project developer to choose how many parts 
>> if
>> appropriate for their project. And I don't see any problems with doing 
>> that.
>
> Ok. But then I think we most allow: ">  0.3.4+".

Sorry, can you be more specific on what you mean by that?


>>> I see three options:
>>>
>>> * One package for all platforms
>>> * Include the platform in the package name and in the orbspec
>>> * Have a sub path (on the server) for every platform, i.e.:
>>>
>>> dorbit.org/orbs/linux/dwt-1.3.2.orb.zip
>>>
>>
>> I think the first option makes the most sense, and the second option 
>> sounds
>> like it would be a sensible workaround when someone really wants things
>> separate.
>
> There's always this issue with downloading files you don't need.
>

True, so I guess there should be a way to do platform-specific packages.


>> I'm not sure I like the third option because that wouldn't work for
>> repositories that aren't explicity chosen unless you made it a standard
>> naming system, but then that creates extra requirements for the server, 
>> and
>> then how do you know whether to GET from the platform-specific directory 
>> or
>> the platform-independent/cross-platform directory, etc. And what about
>> having a windows package and a combination OSX/linux package? It's 
>> workable,
>> but I'm not sure it's worth it in light of the first two options.
>
> I don't understand what you mean.
>

Suppose there's a (possibly default) repository:

http://www.super-d-repos.com/joesmith/

And the user runs:

$ orb install foo 1.2.3

What url does orb try to retreive?:

http://www.super-d-repos.com/joesmith/foo-1.2.3.zip
http://www.super-d-repos.com/joesmith/linux/foo-1.2.3.zip
http://www.super-d-repos.com/joesmith/multiplatform/foo-1.2.3.zip

Etc...

We'd need to have some well-designed standard for how that works.

I'm not sure we should require the repo actually be specified as:

http://www.super-d-repos.com/joesmith/linux/

Because that would be platform-dependent and require extra platform-handling 
code for the orbspec author. And it would be different for different repos. 
Come to think of it, the same issue applies to making the platform part of 
the package name. So maybe a good path-based system would be better.

This makes me think of another question: Once different comression formats 
are allowed, if someone uses orb to download/install a package, how does orb 
know whether to grab a .zip, or a .bz2, or a .7z, etc?


> Why would you have one package for Windows and one for osx/linux? Or have 
> I misunderstood something.
>

If, for example, the author wants the windows package to be separate because 
it (hypothetically) has a lot of differences from posix, but the different 
posix versions are very similar so may as well be in the same package.


>> Although, on second thought, if a good system for the third option could 
>> be
>> worked out, then maybe that would be best. (But maybe just not for the
>> initial release?)
>
> An other issue with one package for all platforms is when building the 
> package. The developer first needs to build the package on one platform, 
> then move the package to all other platforms and rebuild the package. The 
> tool needs to open the package and add files for the other platforms.
>

Good point. Although, of course, that only applies to binary packages. So 
yea, it seems that both multi-platform and platform-specific packages should 
be supported. And there needs to be some system for dealing with that.


>>
>> So it's purely for creating a package?
>
> Yes, the "files" field is just when creating the package.
>

Ok.


> As I said before, "libraries" and "executables" are what actually should 
> be installed.
>

This is another thing I'm unclear on, the nature of "installing". It almost 
sounds like you're saying that all the files in the package other than 
"libraries" and "executables" are just not used for anything and just thrown 
away. I doubt that's it though.

Is this how you're envisioning it?:

1. User says "orb install foo".

2. Orb downloads and extracts the foo package to a temp directory.

3. Orb invokes the build process to build foo (BTW, how is that "how to 
build" commandline string  specified in the orbspec? The closest thing I see 
is the "build" field, but it looks like that's just the name of the tool 
used. An actual command line string is going to be needed.)

4. Orb copies the files listed in "libraries" and "executables" from the 
temp dir to their permanent location.

5. Temp dir is cleared.

If so, how do you account for non-compiled source-based libraries? Include 
the *.d files in "libraries"?


> This is what I'm thinking and is planning for Orbit and my build tool 
> Dake. Dake will be able to generate an orbspec file, because, as you said, 
> the build tool will have all the necessary information. Then you don't 
> have to duplicate any information.

Hmm, I guess "files" isn't so bad.


>> Ouch. I have to say, I don't like that at all. The way I see it, this is 
>> one
>> of the two primary responsibilities of a package manager for a language 
>> (the
>> other being automatic dependency handling). Without it, we're not much 
>> ahead
>> of where we are right now.
>
> If you have suggestions I'm listening.
>

Lemme think about it a bit and get back to you.


>> That doesn't work: If orb is managing the packages, then how is
>> dake/drake/etc supposed to know what path to include for -I? Orb knows 
>> where
>> it's sticking packages. The build systems don't know that.
>
> The build tool invokes Orbit and asks about the include path.
>

I think it'd be nicer if invoking a process wasn't needed for that. 
(Although it could just be done in the "configure" step, but it'd be nicer 
if orb could just handle it without the buildscript needing to worry about 
it.)


>> This also reminds me of another issue: For D libraries, the
>> orbfile/orbspec/metadata/whatever needs to give the relative path for the
>> base include directory.
>
> Something like that. Or if given full path the tool will assume the 
> current directory is the base directory and remove that from the path. For 
> example:
>
> $ cd ~/projects/d/foo
>
> Contains:
>
> main.d
> bar/foobar.d
>
> # foo.orbspec
> files << ["~/projects/d/foo/main.d", "~/projects/d/foo/bar/foobar.d"]
>
> $ orb build foo
>
> Since I run the "orb build" command in the "~/projects/d/foo" directory 
> the tool will just remove that from the paths when including the files in 
> the package.
>

What about projects that don't use "{project_root}" as the include 
directory? My projects, for instance, typically use "{project_root}/src".


>> I think that's very, very bad. I should be able to compile something with 
>> a
>> different version of DMD without reinstalling everything. "dvm use xxx"
>> should *just work*, just as it already does now.
>
> The whole point is to be able to have different packages installed with 
> different compilers. There could be a command for moving over packages 
> from one DMD installation to another.
>

Seems unnecessary.


> Say I'm using DMD 2.053, then installing package "foo". Then I'm 
> installing DMD 2.054 and switching to it. Say also that package "foo" 
> doesn't work with DMD 2.054, what happens when you switch to 2.054?
>

The same thing as if you manually did:

$ dvm use 2.054
$ rdmd -I~/proj/foo-1.2.3/includes myApp.d

It just won't work.


>> Orb manages different versions of arbitrary packages. DVM manages 
>> different
>> versions of one specific package. If they're so different despite all 
>> that,
>> then either Orbit isn't generalized enough, or DVM is too specialized.
>
> DVM is very specialized, yes. The installation and switching of D 
> compiler, using DVM, is very specialized. DMD requires wrappers, 
> manipulating the PATH or registry. None of the Orbit packages require 
> this.
>

What about "orb use" on binary packages?


> Orbit would require loads of special cases to have DVM built-in. On your 
> description DVM and Orbit sound very similar but how they actually work 
> when installing and switch compilers are very different. You should know 
> this, you ported DVM to Windows.
>

What I'm saying is that maybe they shouldn't be so different.


> The current compiler decides what packages are available/installed, that 
> wouldn't work if the compiler package would just be a regular orb package.
>

I don't believe that the current compiler *should* decide what packages are 
available/installed. Maybe there could be some optional magic to warn when 
using a compiler/package combination that isn't known to work. But aside 
from that, I really think these should be orthogonal.