Groups | Blog | Home
all groups > sql server (microsoft) > february 2006 >

sql server (microsoft) : data type, real


zamdrist NO[at]SPAM gmail.com
2/28/2006 7:24:20 AM
Why is it when I do a 'return all rows' from a table that contains a
data type column of Real I get 0.031, but when I Select All from Query
analyzer I get 3.0999999E-2?

I'm trying to perform a caluculation like:

Cost = (tblAreaCodeRateMatrix.Rate /60) * Duration

Where Rate is Cost/Minute and Duration is a whole number representing
Seconds. I'm unsure if I am getting the results I would imagine I
should get.

Oddly enough here are two different results:

(Rate/60) * Duration

(.022 / 60) * 3827 = 1.403
(3.0999999E-2 / 60) * 2966 = 1.53

Thoughts? Should I store .022 or .0XX as a Real number?
Barry
2/28/2006 10:06:35 AM
What are you using to "return all rows" ??
Zamdrist
2/28/2006 10:34:14 AM
Enterprise Manager
Barry
2/28/2006 11:34:08 AM
Hmm - I get the same problem too. I can only guess it's a bug?

In any case I would stick with Query Analyzer.

Barry
Zamdrist
3/3/2006 11:23:25 AM
A datatype of Decimal with a Precision of 18 and Scale of 3 seemed to
work best, instead of Real.
AddThis Social Bookmark Button