[Issue 12164] Function returning ptrdiff_t.min in 64-bit returning 0 when -O is set.

[d-bugmail at puremagic.com]

https://d.puremagic.com/issues/show_bug.cgi?id=12164



--- Comment #7 from safety0ff.bugz <safety0ff.bugz at gmail.com> 2014-03-31 07:33:59 PDT ---
(In reply to comment #6)
> If someone can identify exactly which instructions are wrong, and what they
> should be, I can probably fix it fairly easily.  Otherwise it will have to
> wait, I don't have the energy to go digging through the assembly right now.

I don't think the solution is as simple as you think (it's not a simple REX
issue):
Un-optimized (correct) assembly:
    55                    push    RBP
    48 8B EC              mov    RBP,RSP
    48 83 EC 10           sub    RSP,010h
    48 B9 04 00 00 00 00 00 00 00     mov    RCX,4
    33 C0                 xor    EAX,EAX
    F3                    rep
    A6                    cmpsb
    75 05                 jne    L1D
    48 31 C0              xor    RAX,RAX
    C9                    leave
    C3                    ret
L1D:    48 B8 00 00 00 00 00 00 00 80     mov    RAX,08000000000000000h
    C9                    leave
    C3                    ret

Incorrect -O assembly:
    55                    push    RBP
    48 8B EC              mov    RBP,RSP
    48 83 EC 10           sub    RSP,010h
    48 B9 04 00 00 00 00 00 00 00     mov    RCX,4
    33 C0                 xor    EAX,EAX
    F3                    rep
    A6                    cmpsb
    74 05                 je    L1D
    1B C0                 sbb    EAX,EAX
    83 D8 FF              sbb    EAX,0FFFFFFFFFFFFFFFFh
L1D:    83 F8 01              cmp    EAX,1
    19 C0                 sbb    EAX,EAX
    31 C0                 xor    EAX,EAX
    48 8B E5              mov    RSP,RBP
    5D                    pop    RBP
    C3                    ret

As you can see, it invariantly returns zero after doing the memcmp.

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------