High memory usage in vibe.d application
Anton Fediushin
fediushin.anton at yandex.com
Fri Jun 29 09:34:45 UTC 2018
Hello, I'm looking for an advice on what I am doing wrong.
I have a vibe.d-based program, which connects to an audio stream
and gets name of the song currently playing. For that, I wrote
the following code:
```
@safe string nowPlaying(string url) {
import vibe.core.stream;
string r;
url.requestHTTP(
(scope req) {
req.headers.addField("Icy-MetaData", "1");
},
(scope res) {
auto metaint = res.headers.get("icy-metaint").to!int;
auto buffer = new ubyte[metaint];
res.bodyReader.read(buffer, IOMode.all);
auto lengthBuff = new ubyte[1];
res.bodyReader.read(lengthBuff, IOMode.all);
auto dataBuffer = new ubyte[lengthBuff[0] * 16];
res.bodyReader.read(dataBuffer, IOMode.all);
r = dataBuffer.map!(a =>
a.to!char).split('\'').drop(1).front.array.idup;
}
);
return r;
}
```
And I call it with a timer every 10 seconds:
```
string now_playing;
10.seconds.setTimer(() {
now_playing = nowPlaying(stream);
}, true);
```
This code worked fine for 8 or so hours and then got killed by
docker because of a limit of 64MB of RAM. I executed the same
code on my machine and saw resident set size growing in
real-time. Blaming GC (as people usually do) I changed the code
to use std.experimental.allocator instead:
```
@safe string nowPlaying(string url) {
import vibe.core.stream;
import std.experimental.allocator;
string r;
url.requestHTTP(
(scope req) {
req.headers.addField("Icy-MetaData", "1");
},
(scope res) {
auto metaint = res.headers.get("icy-metaint").to!int;
auto buffer = theAllocator.makeArray!ubyte(metaint);
scope(exit) theAllocator.dispose(buffer);
res.bodyReader.read(buffer, IOMode.all);
auto lengthBuffer = theAllocator.makeArray!ubyte(1);
scope(exit) theAllocator.dispose(lengthBuffer);
res.bodyReader.read(lengthBuffer, IOMode.all);
auto dataBuffer = theAllocator.makeArray!ubyte(lengthBuffer[0]
* 16);
scope(exit) theAllocator.dispose(dataBuffer);
res.bodyReader.read(dataBuffer, IOMode.all);
r = dataBuffer.map!(a =>
a.to!char).split('\'').drop(1).front.array.idup;
}
);
return r;
}
```
And somehow, it got *worse*. Now my program gets killed every 3
hours. How is that possible? Am I missing something?
Some screenshots of CPU/Memory usage:
1. These are metrics of a whole cluster, program is started at
around 8:00 and gets killed after 16:00
https://imgur.com/a/IhHvOt4
2. These are metrics of an updated program which uses
std.experimental.allocator. https://imgur.com/a/XBchJ7C
More information about the Digitalmars-d-learn
mailing list