Double value is rounded to unexpected value: 2.5 -> 2 instead of 3

ag0aep6g via Digitalmars-d-learn digitalmars-d-learn at puremagic.com
Fri Jul 7 12:00:07 PDT 2017


On 07/07/2017 08:29 PM, alex_ca wrote:
> I'm having trouble understanding why in some cases a double value will 
> be rounded up and other times down, for the same code. Here's a snippet 
> with code I tried to debug:
> 
>    int getNumberOfStitchesForRowLength(double rowLength)
>    {
>      writeln("input ", rowLength, " ", currentGauge.stitch_gauge, " ", 
> currentGauge.gauge_length);
> 
>      writeln("stitches: ", (rowLength * currentGauge.stitch_gauge) / 
> currentGauge.gauge_length,  " -> " , ((rowLength * 
> currentGauge.stitch_gauge) / currentGauge.gauge_length).roundTo!int);
> 
>      double end = 2.5;
>      double start = 0;
>      writeln("I expect: ", ((abs(end-start)*10)/10).roundTo!int);
> 
>      return ((rowLength * currentGauge.stitch_gauge) / 
> currentGauge.gauge_length).roundTo!int;
> 
>    }
> 
> And here's some output from that:
> input 2.5 10 10
> stitches: 2.5 -> 2
> I expect: 3
> 
> OR, similarly, I get
> 
> input 3.5 10 10
> stitches: 3.5 -> 3
> I expect: 4
> 
> However, it works as I would expect for one value:
> 
> input 1.5 10 10
> stitches: 1.5 -> 2
> I expect: 2
> 
> I would appreciate some ideas for why I see this seeming inconsistency. 
> Or if someone can share how I can further debug this.

Works for me when I plug in the exact values:

----
import std.stdio;
import std.math: abs;
import std.conv: roundTo;
import std.math;

struct Gauge
{
     double stitch_gauge;
     double gauge_length;
}

Gauge currentGauge = Gauge(10, 10);

int getNumberOfStitchesForRowLength(double rowLength)
{
     writeln("input ", rowLength, " ", currentGauge.stitch_gauge, " ",
         currentGauge.gauge_length);

     writeln("stitches: ",
         (rowLength * currentGauge.stitch_gauge) / 
currentGauge.gauge_length,
         " -> " ,
         ((rowLength * currentGauge.stitch_gauge) /
             currentGauge.gauge_length).roundTo!int);

     return ((rowLength * currentGauge.stitch_gauge) / 
currentGauge.gauge_length).roundTo!int;
}

void main()
{
     foreach (x; [1.5, 2.5, 3.5])
     {
         getNumberOfStitchesForRowLength(x);
     }
}
----

Prints:

----
input 1.5 10 10
stitches: 1.5 -> 2
input 2.5 10 10
stitches: 2.5 -> 3
input 3.5 10 10
stitches: 3.5 -> 4
----

As expected, right?

Could it be that the inputs aren't exactly the numbers you think they 
are? For example, when you change one the 10s to 9.9999999, it still 
prints as "10", but the results are different.


More information about the Digitalmars-d-learn mailing list