Are Fibers just broken in D?

[Byron Heads]

On Tuesday, 24 April 2018 at 13:36:48 UTC, Steven Schveighoffer 
wrote:
> On 4/24/18 5:11 AM, bauss wrote:
>> On Tuesday, 24 April 2018 at 07:58:01 UTC, Radu wrote:
>>> On Tuesday, 24 April 2018 at 00:46:39 UTC, Byron Heads wrote:
>>>>
>>>> Fibers on Win32 have a memory leak for sure:
>>>>
>>>> import core.thread : Fiber;
>>>>
>>>> void main() {
>>>>
>>>>     foreach(ulong i; 0..99_999) {
>>>>         auto foo = new Foo();
>>>>         foo.call();
>>>>         foo.call();
>>>>     }
>>>> }
>>>>
>>>>
>>>> class Foo : Fiber {
>>>>     this() {
>>>>         super(&run);
>>>>     }
>>>>
>>>>
>>>>     void run() {
>>>>         Fiber.yield();
>>>>     }
>>>> }
>>>>
>>>>
>>>> Running this with -m64 on windows runs without a problem, 
>>>> but with -m32 it failes aith a Memory Allocation failed 
>>>> error.
>>>
>>> This is not a fiber issue but a more memory management issue. 
>>> Your run out of address space on win32, the GC will not 
>>> always collect all those 99999 fibers that you allocate in 
>>> that loop. As an exercise replace `auto` with `scope` like 
>>> `scope foo = new Foo();` in that loop - you should see 
>>> different results.
>
> This shouldn't be a requirement, the 32-bit GC is generally not 
> this bad.
>
>>>
>>> The issue boils down to the call to VirtualAlloc that the 
>>> fiber constructor makes, which fails as Windows will not 
>>> allocate any more virtual pages for the process. If you 
>>> really want that many fibers you should reconsider 32 bit, 
>>> and definitely use a different allocation strategy.
>> 
>> And in the end of the day it makes no sense to have that many 
>> fibers as they would probably perform terrible.
>
> Let's not forget though, that he's executing the fiber 
> completely within the loop. It should be done and collected.
>
> This is not the case of executing 100,000 concurrent fibers, 
> but executing 100,000 *sequential* fibers. It should work just 
> fine.
>
> -Steve

Correct, in a normal run of my system there maybe 10-20 fibers 
max alive. I was using threads only before, but found the system 
to execute jobs in a balanced way. But using a few threads to 
process Fibers keep in a queue balances out the work evenly. It 
is also easier to track down bugs by just using 1 thread to 
process the fiber pool.

I would love to use dip1000, Allocators, and shared. But none of 
that stuff really works beyond trivial examples. (Allocators 
probably works fine, but there are forum post about it changing 
and I dont want to refactor it twice..)

I will start ignoring win32 when win64 doesn't require dealing 
with visual studio installs.
Also I have a feeling a client will ask for it.