language feature usage statistics

Dennis dkorpel at
Sun Oct 20 22:03:22 UTC 2019

On Sunday, 20 October 2019 at 21:29:24 UTC, aliak wrote:
> Which APIs did you use to get all d project links? Does dub 
> provide something?

There might be an API, but I simply parsed the html pages.
First I get the identifiers of all packages:
string page = 
string[] result;
foreach(m; page.matchAll(regex(`packages/([a-zA-Z0-9_-]+)`)))
     result ~= m[1];

Then I parse the package pages for the repository link with htmld:

import html; //
string getRepo(string packageName) {
     string page = 
     auto doc = createDocument(page);
     if (auto p = doc.querySelector("#repository")) {
         if (auto m = p.html.matchFirst(`href="([^"]+)`)) {
             return m[1].text;

> And curious, were you rate limited by github (i'm assuming this 
> was the work of a for loop?).

I wouldn't have been surprised if I got a timeout for cloning 
1600 repositories in succession, but I didn't. (I suppose the 
same happens when installing your average NPM package, lol)

More information about the Digitalmars-d mailing list