Why do floats start good and end bad?

Question

Floats are known for not being accurate: At some point the decimals will turn out incorrectly.

The typical explanation is that the decimal and the binary system are not fully compatible.

The digressions seem to get worse, though, the farther you read into the decimals (at least my little experiment down there seems to suggests that).

So why is it, if the bases are not compatible, that the results are not all over the place but only get consistently bad after about the 15th digit?

https://code.sololearn.com/cpojI69bhp9p/?ref=app

Answer 1

Rounding basically.
You know how you can write decimals in scientific notation? Like 1234 = 1.234 * 10^3.

In that example the "10" is the base, "3" is the exponent and ".234" the mantissa.
I'm simplifying, but floats are basically numbers in binary scientific notation, with a fixed size mantissa and exponent.
For single precision floats you have 8 bits of exponent and 23 bits of mantissa (and 1 sign bit).

That means that small numbers can be represented very accurately and large numbers have larger gaps between them.
Thats because as the exponent gets larger, the dot shifts to the right so to speak, so of those 23 mantissa bits more of them will be used for the integer part of the number and less for the fractional part.
So that 15 digit barrier you are only seeing because you use small numbers. Very large floats may even be 1 or 2 apart from another.

The float standard is called IEEE 754, you can look it up :)

Answer 2

comment too long...
So also, with any fixed size decimal you can never represent 1/3 or 1/7 because the stuff after the decimal point goes on forever.
So at some point you have to truncate or round the number and that also happens with binary floats.

Answer 3

[meta, support code]
For the purposes of correlation to Schindlabua's answer, Javascript floats work in exactly the same way* as Python:
https://code.sololearn.com/WYCTOysAGnLv/?ref=app

You can use this to examine epsilon (which varies by magnitude, manifested by skipped numbers), subnormals (abnormally high precision near 0), the bit zones, special floats, repeating values, etc.

* well...there are variations further down but it's a special kind of detail obsession down there--just note that identical calculations can vary by 2*epsilon on "this config" but "not that one", and only when you hit a certain hardware threshold a hundred decimal places down... (all langs, no joke)