DIP11: Automatic downloading of libraries

[Andrei Alexandrescu]

On 6/15/11 8:33 AM, Steven Schveighoffer wrote:
> On Tue, 14 Jun 2011 09:53:16 -0400, Andrei Alexandrescu
> <SeeWebsiteForEmail at erdani.org> wrote:
>
>> http://www.wikiservice.at/d/wiki.cgi?LanguageDevel/DIPs/DIP11
>>
>> Destroy.
>
> I put this as replies in several threads, but I'll throw it out there as
> its own thread:
>
> * You already agree that having the fetching done by a separate program
> (possibly written in d) makes the solution cleaner (i.e. you are not
> infiltrating the code that actually does compiling with code that does
> network fetching).

I agree.

> * I think specifying the entire url in the pragma is akin to specifying
> the full path of a given module on your local disk. I think it's not the
> right place for it, the person who is building the code should be
> responsible for where the modules come from, and import should continue
> to specify the module relative to the include path.

I understand. It hasn't been rare that I would have preferred to specify 
an -I equivalent through a pragma in my D programs. Otherwise all of a 
sudden I needed to have a more elaborate dmd/rdmd line, and then I 
thought, heck, I need a script or makefile or a dmd.conf to build this 
simple script... I don't think one is good and the other is bad. Both 
have their uses.

BTW, Perl and Python (and probably others) have a way to specify paths 
for imports.

http://www.perlhowto.com/extending_the_library_path
http://stackoverflow.com/questions/279237/python-import-a-module-from-a-folder

> * A perfect (IMO) way to configure the fetch tool is by using the same
> mechanism that configures dmd on how to get modules -- the include path.
> For instance -Ihttp://xxx.yyy.zzz/package can be passed to the compiler
> or put into the dmd.conf.

HTTP is not a filesystem so the mechanism must be different. I added a 
section "Command-line equivalent": 
http://www.wikiservice.at/d/wiki.cgi?LanguageDevel/DIPs/DIP11#section10

My concern about using cmdline/conf exclusively remains. There must be a 
way to specify dependencies where they belong - with the source. That is 
_literally_ where they belong!

One additional problem is one remote library that depends on another. 
You end up needing to add K URLs where K is the number of dependent 
libraries. The process of doing so will be mightily annoying - repeated 
failure to compile and RTFMs.

> * DMD already has a good mechanism to specify configuration and you
> would barely have to change anything internally.
>
> Here's how it would work. I'll specify how it goes from command line to
> final (note the http path is not a valid path, it's just an example):
>
> dmd -Ihttp://www.dsource.org/projects/dcollections/import testproj.d
>
> 1. dmd recognizes the url pattern and stores this as an 'external' path
> 2. dmd reads the file testproj.d and sees that it imports
> dcollections.TreeMap
> 3. Using it's non-external paths, it cannot find the module.
> 4. It calls:
> dget -Ihttp://www.dsource.org/projects/dcollections/import
> dcollections.TreeMap
> 5. dget checks its internal cache to see if the file
> dcollections/TreeMap.[d|di] already exists -- not found
> 6. dget uses internal logic to generate a request to download either
> a. an entire package which contains the requested import (preferred)
> b. just the specific file dcollections/TreeMap.d
> 7. Using the url as a key, it stores the TreeMap.d file in a cache so it
> doesn't have to download it again (can be stored globally or local to
> the user/project)
> 8. Pipes the file to stdout, dmd reads the file, and returns 0 for success
> 9. dmd finishes compiling.

Not so fast. What if dcollections depends on stevesutils, to be found on 
http://www.stevesu.ti/ls and larspath, to be found on http://la.rs/path? 
The thing will fail to compile, and there will be no informative message 
on what to do next.


Andrei