[Issue 1786] New: Phobos cannot accommodate multiple stacks

[d-bugmail at puremagic.com]

http://d.puremagic.com/issues/show_bug.cgi?id=1786

           Summary: Phobos cannot accommodate multiple stacks
           Product: D
           Version: unspecified
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: enhancement
          Priority: P2
         Component: Phobos
        AssignedTo: bugzilla at digitalmars.com
        ReportedBy: default_357-line at yahoo.de


Since the current version of Phobos always scans one whole stack (defined as
"stored stack bottom to ESP"), any advanced constructs which require multiple
stacks (like coroutines) are completely impossible to implement without
disabling the entire GC whenever a stackthread is somewhere active, which in
multi-threaded programs might disable the GC completely.

This issue can be fixed by introducing a way to override the phobos-perceived
stack top, thus limiting the GC to the actual stack. The attached patch against
GDC 0.23's Phobos does this.

Another problem is that stacks should be allocated as they're being used;
however, dynamic arrays cannot do that because resizing might require us to
move the stack elsewhere, which causes problems if we're currently using it :)

A solution to this is using memory-mapped files to contain the stack (as they
don't allocate memory unless it's being used, but only reserve some address
space). This leads to another problem: if the entire MmFile stack is registered
with the GC as a Range, the subsequent GC scan causes the operating system to
allocate memory for the whole range of the stack, causing an explosion of
memory usage.

The way to prevent this is to only register the specific range that is
currently in use, After the threads have been stopped and ESPs updated, but
before the garbage collection is run.

For this, a way is required for the program to register a callback with the GC
that is evoked when the garbage collection is being run (this callback must of
course not allocate any memory).

The attached patch adds this also.

The patch has been tested using the stackthreads implementation in
scrapple.tools, which now depends on it. For all I know, it works.

Since it is impossible to solve the problems described above without this
patch, I request it, or some functionally equivalent version, be included in
the official Phobos trunk.

Looking forward to responses and criticism,
 --feep (downs).


--