0

Computer can't store floats perfectly, like 1/3 = 0.3333... So, how to overcome this problem?

How to instruct the computer to choose 3 decimal place or round up or even convert a float to integer?

12th Dec 2016, 3:38 PM
Ashish Sharma
Ashish Sharma - avatar
3 Antworten
+ 2
Usually the computer is accurate enough for most purposes. Sometimes you need a "fuzz" factor on comparisons. For example, because errors may accumulate in calculations: x == y Will be false even though to two values should be equal. You may need to resort to: (abs(x - y) / (abs(x) + abs(y)) < fuzz where abs is the absolute value function in the language of your choice and fuzz is some error margin suited to your application (say 0.0001). Note the division by the sum of the absolute values is needed to scale the comparison. If the values are small, the difference alone may result in a consistently true result. If they are large, the difference alone will give a consistently false result. You should create a function to avoid repeating the code over and over. If the language permits it, you could overload the comparison operator. There maybe a library available to save you the trouble of writing and debugging you own code. Finally, some languages have a 'fuzz' factor built-in and you may adjust it to suit you needs. Often, in beginner's courses, you write a program to calculate how many bills and coins to distribute a given amount with the fewest number of bills and coins. If you use floating point arithmetic you will be short a penny on something like 555555.99 because of the limited precision of floating point variables and accumlated errors. Convert the amount to an integer (multiply by 100) to avoid the issue. In other circumstances that option is not available.
12th Dec 2016, 6:10 PM
Gordon Garmaise
Gordon Garmaise - avatar
+ 1
round((1/3),2)
12th Dec 2016, 4:48 PM
[No Name]
[No Name] - avatar
0
You can use the Fraction module to have exact computations, and then convert back to float.
12th Dec 2016, 9:36 PM
Amaras A
Amaras A - avatar