DUB - call to arms

[H. S. Teoh]

Recently, Nick & some others (including myself) created a Github 
project to discuss the possibility of a dub replacement that has 
a better architecture. In the course of analysing our current 
situation and where we'd like to be, Nick pointed out something 
that immediately strikes me as low-hanging fruit that could 
potentially improve dub's performance significantly.

According to Nick, every dub package carries its own description 
file (sdl/json) that encodes, among other things, its version and 
dependencies.  Apparently, this information resides per-package 
on code.dlang.org, and so every time dub has to resolve 
dependencies for a particular package, it has to download the 
description files of *every version* of the package??  Can 
somebody confirm whether or not this is really the case?

Because if this is true, that means dub is making N network 
roundtrips *per dependency* just to obtain dependency information 
from each package.  This is obviously extremely inefficient, and 
I'll bet that it's a significant source of dub's perceived 
slowness.

But this also immediately suggests a simple solution: upon 
uploading a package to code.dlang.org, the server should collect 
the description files of all versions of that package, and make 
that available somewhere in a per-package global description 
file.  It doesn't need to contain everything in dub.{sdl/json}; 
all it needs is version information and per-version dependency 
lists.  This way, whenever dub needs to resolve dependencies on 
that package, it can obtain all of the information it needs on 
that package with a single HTTP roundtrip.  Of course, it will 
still need to make more roundtrips in order to obtain information 
for sub-dependencies and other dependencies, but if what Nick 
says is true, this should significantly cut down on the total 
number of roundtrips required (O(n) as opposed to O(n*v), where n 
= #packages, v = #versions-per-package).


--T