code.dlang.org reliability update

[Sönke Ludwig]

Am 04.03.2020 um 20:06 schrieb WebFreak001:
> On Monday, 2 March 2020 at 19:17:59 UTC, Sönke Ludwig wrote:
>> As of yesterday, code.dlang.org now points to a more powerful 
>> dedicated server that can host the DUB registry without the danger of 
>> freezing due to excessive swapping - this is what happened on the 26th 
>> last month [1].
>>
>> In addition to that, the server that previously hosted the registry is 
>> now used to run an official mirror, reachable at codemirror.dlang.org. 
>> This will be configured as a built-in fallback server starting with 
>> DMD 2.091.0/DUB 1.12.0 and, at least in theory, will lead to an uptime 
>> of virtually 100%.
>>
>> To make use of the mirror today, it is also possible to configure it 
>> in DUB's settings.json as a custom registry:
>>
>> {
>>    "registryUrls": ["https://codemirror.dlang.org/"]
>> }
>>
>> settings.json is found/needs to be created in %APPDATA%\dub\ on 
>> Windows and in ~/.dub/ on all other systems. The custom entry should 
>> be removed once DUB 1.12.0 is used, to avoid redundant requests in 
>> certain situations.
>>
>>
>> [1]: https://forum.dlang.org/thread/ontwwoxuhnoczcokacwm@forum.dlang.org
> 
> thank you very much for this Sönke! Is throwing so much more RAM (= 
> money) and power (= more money) at it going to be a good solution in the 
> long run though?
> 
> Is there maybe a plan for remaking the registry architecture like adding 
> better supported mirrors and load balancing to it?

The server has a different main purpose, so this is just a nice benefit 
for the registry in that regard. The main issue is that there is some 
kind of GC related memory leak that is hard to debug with the basically 
non-existent tooling. Decoupling the package scanning from the web 
frontend into separate processes will play a major role in making that 
much less of an issue, though.

There is at least one fully supported mirror now and the cache based 
mirror by Sebastiaan could also be added as a reliable (albeit partial) 
fallback. So in that regard, at least the pressure for taking more 
involved steps has been mostly relieved now.

But making changes to the general architecture - be it in the form of 
adding load-balancing, or more deeply rooted, such as using GIT for 
storing and distributing the package index - is something that still is 
definitely desirable to push forward.