Fiber pool?
Steven Schveighoffer
schveiguy at gmail.com
Fri Mar 25 02:44:11 UTC 2022
Went through some interesting debugging with someone on Discord.
Long story short -- Fibers manage a resource that is limited. On Linux,
the maximum number of concurrent fibers you can have by default is 2^15.
In practice, this means you really shouldn't be depending on the GC to
clean up your fibers for you. The person on discord had a simple test:
```d
void main () {
import std.stdio;
import core.thread;
import core.memory;
int n = 0;
foreach (a; 0..10) {
foreach (tick; 0..100_000) {
auto gt1 = new Fiber(() {
foreach (i; 0..5) {
n += 10;
Fiber.yield;
}
});
auto gt2 = new Fiber(() {
foreach (i; 0..5) {
n += 10;
Fiber.yield;
}
});
while (gt1.state != Fiber.State.TERM && gt2.state !=
Fiber.State.TERM) {
gt1.call;
gt2.call;
}
assert(gt1.state == Fiber.State.TERM && gt2.state ==
Fiber.State.TERM);
}
GC.stats.writeln;
}
writeln(n);
}
```
This results in a coredump after about 3 writes of the GC stats on my
system.
What ends up happening is the number of fibers goes up to the limit, and
the OS runs out of mmap slots in the kernel.
My recommendation to him is not to manage fibers using the GC, but to
proactively destroy them.
But shouldn't D's standard library have a way to do this easily? Like
some sort of fiber pool you can use to manage one-use fibers (that get
returned to the pool when terminated)? It also might be able to limit
fiber use so instead of an ugly abort OS error, you get a nice message.
Is there something like this already on code.dlang.org?
-Steve
More information about the Digitalmars-d-learn
mailing list