Invalid assembler comparison

Etienne Cimon via Digitalmars-d-learn digitalmars-d-learn at puremagic.com
Fri Oct 23 08:17:42 PDT 2015


Hello,

I've been trying to understand this for a while now:

https://github.com/etcimon/botan/blob/master/source/botan/math/mp/mp_core.d#L765

This comparison (looking at it with windbg during cmp operation) 
has these invalid values in the respective registers:

rdx: 9366584610601550696
r15: 8407293697099479287

When moving them into a ulong variable with a mov [R11], RDX 
before the CMP command, I get:
RDX: 7549031027420429441
R15: 17850297365717953652

Which are the valid values.

Any idea how these values could have gotten corrupted this way? 
Is there a signed integer conversion going on behind the scenes?


More information about the Digitalmars-d-learn mailing list