Double value is rounded to unexpected value: 2.5 -> 2 instead of 3

alex_ca via Digitalmars-d-learn digitalmars-d-learn at puremagic.com
Fri Jul 7 11:29:57 PDT 2017


Hi,

I'm having trouble understanding why in some cases a double value 
will be rounded up and other times down, for the same code. 
Here's a snippet with code I tried to debug:

   int getNumberOfStitchesForRowLength(double rowLength)
   {
     writeln("input ", rowLength, " ", currentGauge.stitch_gauge, 
" ", currentGauge.gauge_length);

     writeln("stitches: ", (rowLength * currentGauge.stitch_gauge) 
/ currentGauge.gauge_length,  " -> " , ((rowLength * 
currentGauge.stitch_gauge) / 
currentGauge.gauge_length).roundTo!int);

     double end = 2.5;
     double start = 0;
     writeln("I expect: ", ((abs(end-start)*10)/10).roundTo!int);

     return ((rowLength * currentGauge.stitch_gauge) / 
currentGauge.gauge_length).roundTo!int;

   }

And here's some output from that:
input 2.5 10 10
stitches: 2.5 -> 2
I expect: 3

OR, similarly, I get

input 3.5 10 10
stitches: 3.5 -> 3
I expect: 4

However, it works as I would expect for one value:

input 1.5 10 10
stitches: 1.5 -> 2
I expect: 2

I would appreciate some ideas for why I see this seeming 
inconsistency. Or if someone can share how I can further debug 
this.

Thanks!

I'm using DMD32 D Compiler v2.074.1 on Windows


More information about the Digitalmars-d-learn mailing list