[Issue 17914] [Reg 2.075] Fibers guard page uses a lot more memory mappings

d-bugmail at puremagic.com d-bugmail at puremagic.com
Thu Oct 26 21:41:10 UTC 2017


https://issues.dlang.org/show_bug.cgi?id=17914

--- Comment #6 from Steven Schveighoffer <schveiguy at yahoo.com> ---
(In reply to briancschott from comment #5)
> That's not entirely accurate. I found this because the range implementation
> for a complex multi-dimensional tree structure was implemented as a
> std.concurrency.Generator.

Right, the issue here is not so much that you need a certain number of
concurrent fibers, it's that the limited resource (64K items per process) is
tied to memory management (essentially infinite number of items per process,
and no requirement to release).

It's the same argument as to why File is ref-counted with malloc and not a
class.

So toy examples may fail, large concurrent webservers may fail (but arguably,
those can be tuned to work), OR reasonable innocuous code may fail if it falls
into the right trap using GC-managed data.

Worth noting that simply turning off GC collections can trigger this error.

A heavy-handed solution might be that if there is a limit on Fiber creation,
then you should have an object to manage them. But that would be a drastic
design change.

Another possibility is that we make a type-change to track the "good" behavior
and the "bad" behavior separately. For example, SafeFiber, which does the guard
page. Then at least it's opt-in.

(In reply to Martin Nowak from comment #4)
> Why isn't the number of guard pages a problem on Windows? That kernel also
> needs to keep track of lots of memory regions.

The limit is 64k *per process*, so obviously, the kernel can handle quite a bit
more.

Worth noting that Windows has a much different memory layout than Unix.

> If we really need a comprise here, I'd definitely increase the stack size
> further, e.g. to 64KiB (vibe.d's default).

That's also a possible option. We could do that along with the reversal of the
guard page.

--


More information about the Digitalmars-d-bugs mailing list