GDC generates invalid assembly around fiber yield operations (Not re-reading data from clobberedd memory to registers)

[Liran Zvibel via D.gnu]

Hi,

I forgot to mention is before, I'm running with GDC commit 
b022dd4cac195d85e9c3a6a37f2501a07ade455a from April 7th based on 
GCC 4.9.

Does anyone have experience compiling (and then running) Fiber 
based applications with GDC?
What are you doing?

Is there a plan to add support for @attribute("returns_twice") to 
GDC?

Thanks!

Liran

On Sunday, 3 May 2015 at 18:45:36 UTC, Liran Zvibel wrote:
> Hi,
>
> We are trying to port a large fiber based application to GDC.
> Our application works well when compiled with DMD with 
> optimizations.
> It fails very quickly with GDC (even before we tried 
> optimizations), and we were able to narrow it to how GDC treats 
> yields.
>
> Please look at the following small program (minor changes from 
> the core.thread.Fiber example):
>
> import std.stdio;
> import core.thread;
>
> enum numAddition = 3;
> long globalSum = 0;
>
> class DerivedFiber : Fiber
> {
>     int index;
>     this(int _index)
>     {
>         index = _index;
>         super( &run );
>     }
>
> private :
>     void run()
>     {
>         foreach(int k; 0 .. numAddition) {
>             globalSum += otherFunc();
>             writefln("DerivedFiber(%d) iteration %d globalSum 
> is %d", index, k, globalSum);
>         }
>     }
>
>     long otherFunc() {
>         yield();
>         return index;
>     }
> }
>
> int main()
> {
>     Fiber derived1 = new DerivedFiber(1);
>     Fiber derived2 = new DerivedFiber(2);
>
>     foreach(j; 0 .. (numAddition+1)) {
>         derived1.call();
>         derived2.call();
>     }
>
>     assert(globalSum == (1+2)*numAddition);
>     return 0;
> }
>
>
> And the following output when compiling first with DMD and then 
> with GDC:
>
> bash-4.3# dmd -release -O fiber.d -ofdfb
> bash-4.3# ./dfb
> DerivedFiber(1) iteration 0 globalSum is 1
> DerivedFiber(2) iteration 0 globalSum is 3
> DerivedFiber(1) iteration 1 globalSum is 4
> DerivedFiber(2) iteration 1 globalSum is 6
> DerivedFiber(1) iteration 2 globalSum is 7
> DerivedFiber(2) iteration 2 globalSum is 9
> bash-4.3# /opt/gdc/bin/gdc  -ggdb fiber.d -ogfb
> bash-4.3# ./gfb
> DerivedFiber(1) iteration 0 globalSum is 1
> DerivedFiber(2) iteration 0 globalSum is 2
> DerivedFiber(1) iteration 1 globalSum is 2
> DerivedFiber(2) iteration 1 globalSum is 4
> DerivedFiber(1) iteration 2 globalSum is 3
> DerivedFiber(2) iteration 2 globalSum is 6
> core.exception.AssertError at fiber.d(41): Assertion failure
> [ Removed the very simple stack...]
>
>
> When looking at the generated assembly, it's very easy to see 
> that the value of 'globalSum' is read to register before the 
> call to `otherFunc` (and also does not refresh it across the 
> 'foreach'). The problem is that otherFunc calls 'yield' which 
> changes context and causes 'globalSum' to be updated.
> GDC has to know that after `yield` is called memory is 
> potentially clobbered, and re-read memory back to registers.
>
> I tried some things that did not help:
> - declaring 'globalSum' as 'shared' or '__gshared'.
> - declaring 'globalSum' as 'static' inside DerivedFiber, also 
> tried adding 'shared' or '__gshared'.
> - adding 'asm  {"nop;" :  :  : "memory" ;}' after the 'yield'.
> - defining 'otherFunc' with @attribute("returns_twice") -- got 
> 'error: unknown attribute returns_twice'.
>
> Two things did work, but are unpractical when porting a big 
> application (over 100k loc) that heavily relies on Fibers:
> - using a temporary variable to hold the 'otherFunc' return, 
> then add it to 'globalSum'
> - calling core.atomic.otomicOp!"+="(globalSum, otherFunc())
>
>
> We COULD identify all functions that immediately call 'yield' 
> and mark them with @attribute("returns_twice"), but that does 
> not seem to be supported by GDC.
>
> Any ideas?
>
> Thanks!
>
> Liran