0
Why does 1.011-1 return 0.010999999999999899?
When I try subtracting real numbers in different languages (tried Python, Java, Pascal and Visual Basic) I often get results like the one above. I use these kinds of calculations to separate the decimal part of a number in my codes and I believe that making them done right is crucial for a lot of other cases. I'm a beginner and I think that my problem comes from not understanding data types and how they work. Can someone explain this please? https://code.sololearn.com/cGXBCCwBLy6Y/?ref=app
4 Answers
+ 2
The thing is about the way a computer represents numbers. While integers can be sum up as multiple of 2**n (powers of two added).
For fractures the numbers are assembled of sums of 1/2**n. So for most of the numbers only a close approximation can be represented.
https://en.m.wikipedia.org/wiki/Single-precision_floating-point_format
Wikipedia gives a good introduction and links to further reading.
+ 4
The problem it how floating point numbers are stored. They often cant correctly represent every fraction.
What might help you is learning about binary representation of numbers and learning about "IEEE floating point representation"
+ 3
Here is an example in Java:
float has lesser precision than double. BigDecimal has a high precision.
But in all cases you have to round/format the result.
A computer uses binary numbers and we are using decimal numbers. Converting from one system to another works for integer but is not so easy for float/double.
https://code.sololearn.com/cCx87SNRNUqQ/?ref=app
+ 2
https://www.sololearn.com/discuss/1477626/?ref=app