double trouble
John C
johnch_atms at hotmail.com
Sat Feb 25 04:07:08 PST 2006
> I believe the problem is with my scaling code. Any ideas to increase the
> accuracy?
Scratch that. It must be the way I'm converting the digits to a double that
introduces the inaccuracy.
// pDigits is a pointer into a zero-terminated buffer (with the digits
"278").
double v = 0.0;
while (*pDigits) {
v = v * 10 + (*pDigits - '0');
pDigits++;
}
v is now 277999999999999.999990. Then it's scaled using the code previously
posted to get 2.7799999999999998. How would I round this up to the original
2.78?
More information about the Digitalmars-d
mailing list