Orbit - Package Manager - Specification/ideas

Johannes Pfau spam at example.com
Mon Jul 18 09:03:24 PDT 2011


Jacob Carlborg wrote:
>On 2011-07-18 00:26, Nick Sabalausky wrote:
>> "Jacob Carlborg"<doob at me.com>  wrote in message
>> But why should the name of the project and the name of the tool be
>> different? It would be less confusing to have a packager manager
>> named Orb that gets invoked via "orb". (Or a package manager named
>> Orbit that gets invoked via "orbit".
>
>I don't know. It's also easy to separate the library code in one
>package named "orbit" and the application code in on package named
>"orb".
>
>> I think the "metadata" would make sence if, and *only* if, it were
>> "compiled" down to a pure-data format.
>
>Ok, fair enough. That was the initial idea, to have the "metadata"
>file in a pure-data format.
>
>> As for orbfile vs orbspec: I still think these could and should be
>> merged. I don't need a separate buildscript for different actions.
>> *Everything* the buildsystem does is configured in one centralized
>> script (and whatever other files it may or may not choose to
>> delegate out to). I see no reason why the same can't or shouldn't be
>> done with package management.
>
>I guess I can do that.
>
>> If your "orbblah" file doesn't contain the parts needed for orb to
>> auto-create a package, then you just can't have orb auto-create your
>> package. Simple.
>
>Ok.
>
>> Also, I don't see why the name of "orbspec" needs to be prefixed
>> with the package's name. Just like an orbfile, or makefile, or
>> rakefile, etc., you already know what package it's for from what
>> directory/package it's inside of.
>
>The package needs a name. One option is to specify that in the orbspec 
>file. Another option is to specify it in the name of the orbspec file.
>I just thought that it was convenient.
>
>Is it a good idea to take the name directory where the orbspec is
>located?
>
>Note that many of my choices are based on RubyGems.
>
>> Well, any version that's>= 0.3.4 and<  0.4.0
>
>Ok.
>
>> That would make sense. But, it could always be forcefully done by
>> doing it manually without going through orb. Don't know if that's
>> worth worrying about though.
>
>Do you mean if you have direct access to the server where the
>repository is located? I don't think that's something worth worrying
>about. Isn't that the same with all servers you don't own yourself? If
>you have a website on a hosted server they can, if they want to, just
>remove it.
>
>
>> I agree with inferring 0 for any parts not provided.
>
>Good.
>
>> I think it should be up to the project developer to choose how many
>> parts if appropriate for their project. And I don't see any problems
>> with doing that.
>
>Ok. But then I think we most allow: ">  0.3.4+". Or else you cannot
>know if a given version will break the API.
>
>> I think that might be a bit limiting. And keep in mind, just because
>> something is source-only doesn't mean it'll actually work on more
>> than one platform.
>
>As you can read in the specification I do intend to have binary 
>packages. But as I said, that was not my original thought and that's
>why I haven't given it much thought.
>
>Of course it won't work on all platform just because it's source only 
>but if there are source only packages then the problem below doesn't
>exist.
>
>>> I see three options:
>>>
>>> * One package for all platforms
>>> * Include the platform in the package name and in the orbspec
>>> * Have a sub path (on the server) for every platform, i.e.:
>>>
>>> dorbit.org/orbs/linux/dwt-1.3.2.orb.zip
>>>
>>
>> I think the first option makes the most sense, and the second option
>> sounds like it would be a sensible workaround when someone really
>> wants things separate.
>
>There's always this issue with downloading files you don't need.
>
>> I'm not sure I like the third option because that wouldn't work for
>> repositories that aren't explicity chosen unless you made it a
>> standard naming system, but then that creates extra requirements for
>> the server, and then how do you know whether to GET from the
>> platform-specific directory or the
>> platform-independent/cross-platform directory, etc. And what about
>> having a windows package and a combination OSX/linux package? It's
>> workable, but I'm not sure it's worth it in light of the first two
>> options.
>
>I don't understand what you mean.
>
>Why would you have one package for Windows and one for osx/linux? Or 
>have I misunderstood something.
>
>> Although, on second thought, if a good system for the third option
>> could be worked out, then maybe that would be best. (But maybe just
>> not for the initial release?)
>
>An other issue with one package for all platforms is when building the 
>package. The developer first needs to build the package on one
>platform, then move the package to all other platforms and rebuild the
>package. The tool needs to open the package and add files for the
>other platforms.
>
>>>> - Is it really necessary to have separate "build_dependencies" and
>>>> "runtime_dependencies"? And why have both "runtime_dependencies"
>>>> and "orbs"
>>>> instead of just picking one name and sticking with it?
>>>
>>> There is no runtime dependency on something that is statically
>>> linked. Therefore it would be unnecessary to do a permanent
>>> installation on those dependencies. The user could get an option to
>>> either permanently installed these dependencies or to temporarily
>>> install them.
>>>
>>
>> So you mean a statically-linked dependency then? "Runtime
>> dependency" makes me think it's needed at runtime and thus *would*
>> need to be installed. Actually, what you're describing sounds just
>> like a build dependency to me.
>
>Statically-linked dependency is basically the same as build
>dependency. If you have a look at the DVM project page you can see the
>list of build dependencies. The only thing you need to actually run
>the application is zlib (on Poisx), because it's dynamically linked,
>aka a runtime dependency.
>
>> Is there really all that much benefit to doing a temporary install
>> of things temporarally needed anyway? And is it that common of a
>> need? I'm wondering if it's worth the extra complexity.
>
>The package would need access to the dependencies when its build. How 
>should it else be done?
>
>>> The other way around would be possible as well. A package can
>>> depend on a dynamically linked library and use it only through
>>> function pointers. Then the package would only have a runtime
>>> dependency on the library, i.e. it wouldn't be needed when building.
>>>
>>
>> Hmm I think I still don't understand the whole runtime dep vs build
>> dep matter.
>
>If you use, on Posix, dlopen and friends to open a dynamic library you 
>don't need to link to it when building. You call the functions via 
>function pointers, that's how Derelict works, if you have used it.
>
>> That would be up to them. The idea is just that the package metadata
>> is extendable for whatever uses may arise.
>
>Ok, that won't be any problem including.
>
>>>> - What's the point of the fields "files", "libraries" and
>>>> "executables"? Seems like extra work for no real benefit.
>>>
>>> "files" is basically all files that should be put in the package.
>>> "libraries" and "executables" are all libraries and executables that
>>> should be installed (regardless if they are pre-compiled or built
>>> during installation).
>>>
>>
>> So it's purely for creating a package?
>
>Yes, the "files" field is just when creating the package.
>
>> If not, then I'm afraid I still don't understand.
>>
>> If so, then I still don't see the use of "libraries" and
>> "executables". Also I think they should all be optional (if they
>> aren't already) because all that info is already going to exist in
>> the buildscript, and so a lot of people (like me) are going to want
>> to generate the package via their build system rather than
>> duplicating all that work and information just so that the package
>> manager can do it "automatically".
>
>Both the "libraries" and "executables" fields are optional. "summary" 
>and "version" are the only required fields (so far), as you can see on 
>the wiki. As I said before, "libraries" and "executables" are what 
>actually should be installed.
>
>This is what I'm thinking and is planning for Orbit and my build tool 
>Dake. Dake will be able to generate an orbspec file, because, as you 
>said, the build tool will have all the necessary information. Then you 
>don't have to duplicate any information. But, I'm not forcing anyone
>to use my build tool if they want to use Orbit, therefore the fields 
>"libraries" and "executables" are available.
>
>> Other things to think about are: How does the user actually specify
>> the login info? Wouldn't want them in the
>> orbfile/orbspec/metadata/etc.
>
>No, absolutely not. Passing user name and password using HTTP basic 
>authentication on the command line?
>
>> Ouch. I have to say, I don't like that at all. The way I see it,
>> this is one of the two primary responsibilities of a package manager
>> for a language (the other being automatic dependency handling).
>> Without it, we're not much ahead of where we are right now.
>
>If you have suggestions I'm listening.

It would be possible to install libraries into the dmd default search
path. Right now, this would be /usr/include/d/dmd and library files
in /usr/lib on posix, but any path can be used as long as it's included
in dmd.conf. However, this means that only one version of a library can
be installed system-wide, so it's not optimal.

It should also be noted, that linking against a specific library
version can only work well with static libraries. With static libraries
you can give linker a specific path to the library at compile time
(So you can have multiple versions in different directories). Using
different directories with dynamic libraries requires setting
LD_LIBRARY_PATH before executing a program, so this won't work. We'll
have to use the library versioning mechanism that's used by C
libraries (e.g for linux:
http://www.ibm.com/developerworks/linux/library/l-shlibs/index.html ).
I don't know if windows even supports library versioning, but as
windows programs usually don't install dlls globally that's less of a
problem.

>> That doesn't work: If orb is managing the packages, then how is
>> dake/drake/etc supposed to know what path to include for -I? Orb
>> knows where it's sticking packages. The build systems don't know
>> that.
>
>The build tool invokes Orbit and asks about the include path.
>
>> This also reminds me of another issue: For D libraries, the
>> orbfile/orbspec/metadata/whatever needs to give the relative path
>> for the base include directory.
>
>Something like that. Or if given full path the tool will assume the 
>current directory is the base directory and remove that from the path. 
>For example:
>
>$ cd ~/projects/d/foo
>
>Contains:
>
>main.d
>bar/foobar.d
>
># foo.orbspec
>files << ["~/projects/d/foo/main.d", "~/projects/d/foo/bar/foobar.d"]
>
>$ orb build foo
>
>Since I run the "orb build" command in the "~/projects/d/foo"
>directory the tool will just remove that from the paths when including
>the files in the package.
>
>> I think what we need is to have DVM handle dmd.conf/sc.ini
>> separately from the compiler version (which kind of goes along with
>> DVM Issue #9). Orb and DVM need to work together to manage
>> dmd.conf/sc.ini. It's complicated, but I think it's necessary.
>
>I was hoping I didn't need to drag in DVM in this. And I don't won't
>to force anyone using DVM if they want to use Orbit.
>
>> This makes me think of another, too: How does the buildscript know
>> what version of a lib orb has chosen? Since orb packages
>> (rightfully) are allowed to specify a range of versions instead of
>> just one specific version, I think that will be needed.
>
>It doesn't need to know what exact version is chosen, it just needs to 
>know the include paths and libraries to link with. The rest is up to 
>Orbit to handle and decide. The build tool just links with what it 
>received from Orbit.
>
>> I think that's very, very bad. I should be able to compile something
>> with a different version of DMD without reinstalling everything.
>> "dvm use xxx" should *just work*, just as it already does now.
>
>The whole point is to be able to have different packages installed
>with different compilers. There could be a command for moving over
>packages from one DMD installation to another.
>
>Say I'm using DMD 2.053, then installing package "foo". Then I'm 
>installing DMD 2.054 and switching to it. Say also that package "foo" 
>doesn't work with DMD 2.054, what happens when you switch to 2.054?
>
>> Orb manages different versions of arbitrary packages. DVM manages
>> different versions of one specific package. If they're so different
>> despite all that, then either Orbit isn't generalized enough, or DVM
>> is too specialized.
>
>DVM is very specialized, yes. The installation and switching of D 
>compiler, using DVM, is very specialized. DMD requires wrappers, 
>manipulating the PATH or registry. None of the Orbit packages require
>this.
>
>Orbit would require loads of special cases to have DVM built-in. On
>your description DVM and Orbit sound very similar but how they
>actually work when installing and switch compilers are very different.
>You should know this, you ported DVM to Windows.
>
>The current compiler decides what packages are available/installed,
>that wouldn't work if the compiler package would just be a regular orb
>package.
>
>What would be possible it to create a tool that is a front end for DVM 
>and Orbit. I'm currently building Orbit as a library with a thin
>wrapper that is the tool the user will use. I'm planning to rewrite
>DVM in the same way. This also allows to create GUIs and integrate the
>tools into IDEs.
>


-- 
Johannes Pfau



More information about the Digitalmars-d mailing list