How to deal with Number precision in Actionscript?
actionscript, actionscript-3, apache-flex
Solution
This is my generic solution for the problem (I have blogged about this here):
var toFixed:Function = function(number:Number, factor:int) {
return Math.round(number * factor)/factor;
}
For example:
trace(toFixed(0.12345678, 10)); //0.1
- Multiply `0.12345678` by `10`; that gives us `1.2345678`.
- When we round `1.2345678`, we get `1.0`,
- and finally, `1.0` divided by `10` equals `0.1`.
Another example:
trace(toFixed(1.7302394309234435, 10000)); //1.7302
- Multiply `1.7302394309234435` by `10000`; that gives us `17302.394309234435`.
- When we round `17302.394309234435` we get `17302`,
- and finally, `17302` divided by `10000` equals `1.7302`.
Edit
Based on the anonymous answer below, there is a nice simplification for the parameter on the method that makes the precision much more intuitive. e.g:
var setPrecision:Function = function(number:Number, precision:int) {
precision = Math.pow(10, precision);
return Math.round(number * precision)/precision;
}
var number:Number = 10.98813311;
trace(setPrecision(number,1)); //Result is 10.9
trace(setPrecision(number,2)); //Result is 10.98
trace(setPrecision(number,3)); //Result is 10.988 and so on
N.B. I added this here just in case anyone sees this as the answer and doesn't scroll down...
Problem
I have BigDecimal objects serialized with BlazeDS to Actionscript. Once they hit Actionscript as Number objects, they have values like: `140475.32` turns into `140475.31999999999998` How do I deal with this? The problem is that if I use a NumberFormatter with precision of 2, then the value is truncated to `140475.31`. Any ideas?