dub 502 bad gateway

[Sebastiaan Koppe]

On Tuesday, 19 November 2019 at 13:27:31 UTC, Sebastiaan Koppe 
wrote:
> After having had another day where my CI can't build because of 
> gateway errors, this morning I had enough. (I setup an uptime 
> robot so we can see how bad it actually is [1], it checks every 
> 5 min and I just started it.)
>
> [...]
>
> Of course, this is a bandaid solution. Probably dub can be made 
> to do this as well. And likely should. If it were up to me I 
> would go serverless.
>
> [1] https://stats.uptimerobot.com/6mQX4Crw2L
> [2] https://github.com/skoppe/dub-registry-mirror

I have updated the script to cache all routes. This means the 
registry mirror at dub.bytecraft.nl will also work when the 
registry at code.dlang.org is completely down, **if** your 
dependencies have been cached previously.

The cache works in the following manner:

1. if a cache entry exists for a given route and it is no longer 
than 5 minutes old: return that
2. if a cache entry is older than 5 minutes: fetch from upstream, 
cache it and return that
3. if a cache entry is older than 5 minutes and upstream is down: 
reset cache expiry and return from cache
4. if a cache entry doesn't exist: fetch from upstream, cache it 
and return that
5. if a cache entry doesn't exist and upstream is down: return 
error

On top of that there is a 5 second timeout on fetches from 
upstream.

Essentially a cached entry is kept up-to-date (5 minute lag) 
whenever upstream is alive, otherwise stale data is returned if 
available.

Use it like so:

`dub build --registry=https://dub.bytecraft.nl`

Code at 
https://github.com/skoppe/dub-registry-mirror/blob/master/worker.js