High memory usage in vibe.d application

Anton Fediushin fediushin.anton at yandex.com
Fri Jun 29 09:34:45 UTC 2018


Hello, I'm looking for an advice on what I am doing wrong.

I have a vibe.d-based program, which connects to an audio stream 
and gets name of the song currently playing. For that, I wrote 
the following code:

```
@safe string nowPlaying(string url) {
	import vibe.core.stream;
	string r;
	url.requestHTTP(
		(scope req) {
			req.headers.addField("Icy-MetaData", "1");
		},
		(scope res) {
			auto metaint = res.headers.get("icy-metaint").to!int;
			auto buffer = new ubyte[metaint];
			res.bodyReader.read(buffer, IOMode.all);
			
			auto lengthBuff = new ubyte[1];
			res.bodyReader.read(lengthBuff, IOMode.all);

			auto dataBuffer = new ubyte[lengthBuff[0] * 16];
			res.bodyReader.read(dataBuffer, IOMode.all);
			r = dataBuffer.map!(a => 
a.to!char).split('\'').drop(1).front.array.idup;
		}
	);
	return r;
}
```

And I call it with a timer every 10 seconds:

```
string now_playing;
10.seconds.setTimer(() {
	now_playing = nowPlaying(stream);
}, true);
```

This code worked fine for 8 or so hours and then got killed by 
docker because of a limit of 64MB of RAM. I executed the same 
code on my machine and saw resident set size growing in 
real-time. Blaming GC (as people usually do) I changed the code 
to use std.experimental.allocator instead:

```
@safe string nowPlaying(string url) {
	import vibe.core.stream;
	import std.experimental.allocator;
	string r;
	url.requestHTTP(
		(scope req) {
			req.headers.addField("Icy-MetaData", "1");
		},
		(scope res) {
			auto metaint = res.headers.get("icy-metaint").to!int;
			auto buffer = theAllocator.makeArray!ubyte(metaint);
			scope(exit) theAllocator.dispose(buffer);
			res.bodyReader.read(buffer, IOMode.all);
			
			auto lengthBuffer = theAllocator.makeArray!ubyte(1);
			scope(exit) theAllocator.dispose(lengthBuffer);
			res.bodyReader.read(lengthBuffer, IOMode.all);

			auto dataBuffer = theAllocator.makeArray!ubyte(lengthBuffer[0] 
* 16);
			scope(exit) theAllocator.dispose(dataBuffer);
			res.bodyReader.read(dataBuffer, IOMode.all);

			r = dataBuffer.map!(a => 
a.to!char).split('\'').drop(1).front.array.idup;
		}
	);
	return r;
}
```

And somehow, it got *worse*. Now my program gets killed every 3 
hours. How is that possible? Am I missing something?

Some screenshots of CPU/Memory usage:
1. These are metrics of a whole cluster, program is started at 
around 8:00 and gets killed after 16:00 
https://imgur.com/a/IhHvOt4
2. These are metrics of an updated program which uses 
std.experimental.allocator. https://imgur.com/a/XBchJ7C



More information about the Digitalmars-d-learn mailing list