Groups | Blog | Home
all groups > flash actionscript > january 2007 >

flash actionscript : Number precision


dr.Evol
1/3/2007 10:51:06 PM
Hi!

How can I round the number to a particular decimal place?

Example: 0.132456783929 should be 0.1324

Any cool algorithms or existing solutions for that?

Tnx!

Cheers

_AlbertG_
1/3/2007 10:56:11 PM
Multiply the number by 10000 and then round the result.

kglad
1/3/2007 10:57:14 PM
yes, check this forum for a solution. (though you're truncation, not rounding.)

kglad
1/3/2007 10:59:50 PM
that's not going to work, albert.

first, he wants to truncate, not round (if the example is to be believed).
(so, use Math.floor instead of Math.round.)

second, you need to divide the result of your rounding/truncating by your
factor.
_AlbertG_
1/3/2007 11:04:37 PM
Well, He said, "round" altough in the example he typed a truncating number.

kglad
1/3/2007 11:11:37 PM
dr.Evol
1/3/2007 11:12:11 PM
Tnx guys, will check it out...
Well, actually I need both - rounding and truncating. Im writting kind of
financial calculations class, something like FV, PV etc methods...

Truncating could also be done simply by casting to string and cutting the
lenght with substring...

Hmmm....
Math.round(0.123456789 * 10000) / 10000;

.. what if the multiplied number is still far below 0 ? (0.005) Then the round
method will return zero...


_AlbertG_
1/3/2007 11:18:54 PM
Hahahahaha, 2 things...

dr.Evol.
In such case you have to count then numbers you want to round/trucate and
multiply it for more or less "0"

and...

kglad
There was a confusion here, the problem is that I realized that I missed to
divide the number before you write your second message. When you post your
message i've already correct it.
That's because I think we are answering at the same time some questions :D

kglad
1/3/2007 11:20:00 PM
use the appropriate factor. (actually, 10^5 is the wrong factor for your specific question. use 10^4.)

for currency formatting you might check canfieldstudios.com.

dr.Evol
1/4/2007 12:31:24 AM
tnx,

of course I can use different factors but the problem is that I dont know if
the number will be in the range 10^-3,10^-4,10^-5 etc..
I guess I'll put a while loop to determine the range and according to that
use a power of 10 to get the right factor.

Ok..guys..tell me...do you never sleep or you're in the wrong timezone ?! ;)

Good night and tnx!

cheerz

_AlbertG_
1/4/2007 2:13:15 AM
Jeje

Actually, I do.

I'm in GMT -6

Jeckyl
1/4/2007 2:15:27 PM
Please note .. MANY people use the newsgroup interface .. correction NEVER
appear there.

Please .. instead of (or as well as) editing in future, post a reply with
the corrected information.

Otherwise many people will not see your correction (eg. I cannot see it)
--
Jeckyl

AddThis Social Bookmark Button