As designed ☺ [Software]

posted by Helmut Homepage – Vienna, Austria, 2019-07-20 04:12 (1934 d 02:22 ago) – Posting: # 20409
Views: 20,548

Hi zizou,

❝ Recently, because of "floating point error" I failed to use spreadsheet for comparisons of differences of two similar values against certain value (my source data were almost all with two decimal places, but some of them were calculated with "full precision" and I wanted to easily check how many of differences are lower than 0.05, unfortunaly there was a case with the error).


Haha, I know this game and therefore, always round before the comparison. ;-)

❝ Everyone knows, only smart machines don't.


Can’t be blamed. That’s a property of floating point arithmetic (see also IEEE 754). Unless you have powers of 2 (dec→bin: 1→1, 2→10, 4→100, 8→1000, …) you will always have precision issues in the last significant digits. Everything beyond the 15th (dec-) significant figure is just noise – even with 64bit software on a 64bit OS.
Try this in Excel:
Cells A1–A9: 0, 1, … , 8
Cells B1–B9: =2^A1–A9
Cells C2–B9: =B2-B1, B3-B1, …
Format to 16 decimal places. What do you get?

Little bit deeper into [image]:

a <- 5.05
b <- 5
c <- a - b
identical(c, 0.05)
[1] FALSE
sprintf("%.17f", c)
[1] "0.04999999999999982"
a <- 4
b <- 2
c <- a - b
identical(c, 2)
[1] TRUE
sprintf("%.17f", c)
"2.00000000000000000"
identical(c, 2L)
[1] FALSE
is.integer(c)
[1] FALSE
a <- 4L # or as.integer(4)
b <- 2L # or as.integer(2)
c <- a - b
identical(c, 2)
[1] FALSE
sprintf("%.17f", c)
"2.00000000000000000"
identical(c, 2L)
[1] TRUE


GNU Octave 5.1.1.0 (all calculations internally in 64bit double):

>> format long
>> a = 5.05
a = 5.050000000000000
>> b = 5
b = 5
>> b - a
ans = -4.999999999999982e-02
>> a = 4
a = 4
>> b = 2
b = 2
>> a - b
ans = 2


Not even software for symbolic mathematics does better. Maxima 5.35.12.1 / wxMaxima 14.12.1:

(%i1) [a:5.05, b:0.5, a-b];
(%o1) [5.05,5,0.04999999999999982]
(%i2) [a:4, b:2, a-b];
(%o2) [4,2,2]


Only for ne[image]ds: Rmpfr. Doesn’t help. ;-)

library(Rmpfr)
a <- mpfr(5.05, 256)
b <- mpfr(5, 256)
diff(c(b, a))
1 'mpfr' number of precision  256   bits
[1] 0.04999999999999982236431605997495353221893310546875

mpfr(a-b, 32)
1 'mpfr' number of precision  32   bits
[1] 0.05

a <- mpfr(4, 256)
b <- mpfr(2, 256)
diff(c(b, a))
1 'mpfr' number of precision  256   bits
[1] 2


Dif-tor heh smusma 🖖🏼 Довге життя Україна! [image]
Helmut Schütz
[image]

The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

Complete thread:

UA Flag
Activity
 Admin contact
23,280 posts in 4,889 threads, 1,655 registered users;
51 visitors (0 registered, 51 guests [including 11 identified bots]).
Forum time: 05:34 CET (Europe/Vienna)

There is one certainty in drug development
and statistics that one can depend on:
the data are always late.    Scott Patterson and Byron Jones

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5