GC memory fragmentation

[Sebastiaan Koppe]

On Sunday, 11 April 2021 at 09:10:22 UTC, tchaloupka wrote:
> Hi,
> we're using vibe-d (on Linux) for a long running REST API 
> server and have problem with constantly growing memory until 
> system kills it with OOM killer.
>
> [...]
>
> But this is very bad for performance so we've prolonged the 
> interval for example to 30 seconds and now memory still goes up 
> (not that dramatically but still).
>
> [...]
>
> So it wastes a lot of free space which it can't return back to 
> OS for some reason.
>
> Can these numbers be caused by memory fragmentation? There is 
> probably a lot of small allocations (postgresql query, result 
> processing and REST API json serialization).

We have similar problems, we see memory usage alternate between 
plateauing and then slowly growing. Until it hits the configured 
maximum memory for that job and the orchestrator kills it (we run 
multiple instances and have good failover).

I have reduced the problem by refactoring some of our gc usage, 
but the problem still persists.

On side-note, it would also be good if the GC can be aware of the 
max memory it is allotted so that it knows it needs to do more 
aggressive collections when nearing it.