DUB - call to arms

[H. S. Teoh]

On Fri, Apr 26, 2019 at 05:37:35PM -0700, H. S. Teoh via Digitalmars-d wrote:
> On Sat, Apr 27, 2019 at 12:02:58AM +0000, Basile B. via Digitalmars-d wrote:
> [...]
> > For dexed I have a project group that contains the 3 sdl for
> > dmd/druntime/phobos. It's very slow to load... the IDE launches dub
> > 3 times to convert the projects to JSON and it takes 8 to 10
> > seconds.  But none of the sdl contains dependencies, nor configs,
> > build types or whatever...
> > 
> > I have the feeling that dub does something stupid with the files
> > since these 3 repos contains many.
> 
> Shouldn't we be able to just run a profiler on dub to figure out where
> it's spending those 10 seconds? (or about 3 seconds per run.)  Seems
> like something easy enough to identify, even if the problem itself may
> not be so simple to fix.  We should find out whether it's CPU-bound,
> I/O-bound, or network-bound, at the very least.
[...]

OK, so I upgraded my dub to the latest git master, and did a quick and
dirty test.  Init a fresh new project with `-t vibe.d`, accept all
default values (name, license, etc.), then run `time dub -v` to make the
first build.

It took 2 minutes 45 seconds (!).  From a crude visual inspection, most
of the time was spent downloading and compiling the dependencies, so
that seems reasonable, even if unfortunate.

Subsequent runs (remove dub.selections, remove executable, rerun dub -v)
took about 11 seconds or so.  Still rather long, but it's a LOT faster
than the previous version of dub that I tested prior to upgrading to git
master.

The previous version of dub took about 2 minutes 15 seconds for the
first build of a fresh empty vibe.d project, but most of the time
appeared to be spent on the "searching for versions of xyz (1
suppliers)" phase, which I presume was the per-version downloading of
package descriptions.  (The fact that it took 30 seconds less than the
new dub is likely caused by some cached objects from previous vibe.d
builds, so that measurement shouldn't be trusted. I deleted the entire
dub cache before running the test on the new version of dub.)
Subsequent runs (delete dub.selections, rerun dub -v) took about 40
seconds, most of which was spent on the "searching for versions" phase.

So the new dub appeared to have eliminated a good chunk of network
turnaround time, which is good news.

10 seconds is still rather long for an empty project, but from casual
visual inspection of -v output, it appears to still be stuck on the
"searching for versions of vibe.d" phase, which took several solid
seconds.  I'm presuming this is where it's waiting on the network.
Re-running dub without deleting dub.selections eliminates this step, and
turnaround time drops to about 4-5 seconds.  Finally!! It's getting a
lot closer to a serviceable edit-compile-test turnaround time!

So the latest version of dub is looking a lot better than before. Now,
if we could lift some of its functional limitations, we might finally
have an acceptably performant and functional package/build tool.

In the meantime, it would seem that we need to look into why the
"searching for versions" phase takes so long.  Is it just a
network-dependent thing (my network has bogonously slow DNS resolution,
no thanks to my ISP), or is it something that can be fixed in dub
itself?


T

-- 
VI = Visual Irritation