Invalid assembler comparison

Etienne Cimon via Digitalmars-d-learn digitalmars-d-learn at puremagic.com
Fri Oct 23 08:40:12 PDT 2015


On Friday, 23 October 2015 at 15:17:43 UTC, Etienne Cimon wrote:
> Hello,
>
> I've been trying to understand this for a while now:
>
> https://github.com/etcimon/botan/blob/master/source/botan/math/mp/mp_core.d#L765
>
> This comparison (looking at it with windbg during cmp 
> operation) has these invalid values in the respective registers:
>
> rdx: 9366584610601550696
> r15: 8407293697099479287
>
> When moving them into a ulong variable with a mov [R11], RDX 
> before the CMP command, I get:
> RDX: 7549031027420429441
> R15: 17850297365717953652
>
> Which are the valid values.
>
> Any idea how these values could have gotten corrupted this way? 
> Is there a signed integer conversion going on behind the scenes?

I found out that there was an integer conversion going on behind 
the scenes when using jnl. I had to use jnb

http://stackoverflow.com/questions/27284895/how-to-compare-a-signed-value-and-an-unsigned-value-in-x86-assembly


More information about the Digitalmars-d-learn mailing list