Bioequivalence and Bioavailability Forum

Floating point arithmetic, again [Software]

posted by Helmut  – Vienna, Austria, 2020-12-10 20:12 (1372 d 10:44 ago) – Posting: # 22126
Views: 10,777

Dear Ohlbe,

❝ A colleague recently pointed to a strange behaviour in Excel (I have 2013, he has 2016, both give the same result here.

And I have 2000 (don’t laugh).

❝ Type =0.5-0.4-0.1: you'll get the expected result, 0.

Yep.

❝ Now type =(0.5-0.4-0.1): you'll get -2,77556E-17. What ? Ain't that supposed to be the same calculation ?

In principle, yes. Unless we could inspect the source code by M$, difficult to tell what happens with the brackets.

❝ Trying =0=0.5-0.4-0.1: the answer I got was FALSE. WTF, if the answer you get is 0 ? It's not a matter of number of decimals displayed: I tried to add more to the first result, or to switch to scientific notation, but still got 0.

OK, write in A1 =0.5-0.4-0.1 and in B1 =A1=0

1. What do you expect?
   
2. What do you get?

Floating point arithmetic, again (see also this post above).
Try in A1 =16-8-4-2-2, in B1 =(16-8-4-2-2), and in C1 =A1=B1. I bet it “works” as expected, since these numbers can be converted to binary digits without error.

What about ?

options("digits" = 16)
a <- 0.5-0.4-0.1
b <- (0.5-0.4-0.1)
a; b
[1] -2.775557561562891e-17
[1] -2.775557561562891e-17
identical(a, b)
[1] TRUE
c <- 16-8-4-2-2
d <- (16-8-4-2-2)
c; d
[1] 0
[1] 0
identical(c, d)
[1] TRUE

—
Dif-tor heh smusma 🖖🏼 Довге життя Україна! 
Helmut Schütz


The quality of responses received is directly proportional to the quality of the question asked. 🚮
Science Quotes

Complete thread:

• Spreadsheet failures, any recent examples? ElMaestro 2019-07-18 13:43 [Software] 
  • Spreadsheet failures, any recent examples? Ohlbe 2019-07-18 14:32
    • Spreadsheet addiction Helmut 2019-07-18 16:12
      • Nasty beast Ohlbe 2019-07-18 17:30
        • Nasty beast Helmut 2019-07-18 20:26
          • Nasty beast Ohlbe 2019-07-19 11:32
            • Nasty beast ElMaestro 2019-07-19 12:32
              • Decidedly off topic Ohlbe 2019-07-19 13:38
            • OT: R limbo 101 Helmut 2019-07-19 13:00
      • Spreadsheet addiction Shuanghe 2019-07-18 18:59
        • Spreadsheet addiction Helmut 2019-07-18 20:04
          • OT: Spreadsheet addiction Shuanghe 2019-07-19 12:41
            • OT: Spreadsheet addiction Helmut 2019-07-19 14:29
              • OT: Spreadsheet addiction nobody 2019-07-19 15:53
                • OT: Spreadsheet addiction Helmut 2019-07-19 19:32
    • Spreadsheet etc. failures zizou 2019-07-20 00:18
      • As designed ☺ Helmut 2019-07-20 02:12
        • As designed ☺ ElMaestro 2019-07-20 09:08
          • To round or not to round… Helmut 2019-07-20 12:53
            • To round or not to round… ElMaestro 2019-07-20 19:59
              • floating-point math is always more complex than you think it is mittyri 2019-07-20 22:43
  • Spreadsheet failures, any recent examples? Ohlbe 2020-12-10 18:43
    • Floating point arithmetic, againHelmut 2020-12-10 19:12
      • Floating point arithmetic, again ElMaestro 2020-12-10 19:40
        • Floating point arithmetic, again Ohlbe 2020-12-10 20:18
          • Floating point arithmetic, again ElMaestro 2020-12-10 21:38
            • Floating point arithmetic, again Ohlbe 2020-12-10 21:46
              • Floating point arithmetic, again ElMaestro 2020-12-10 22:05
                • Depressed Ohlbe 2020-12-10 23:35
                  • Depressed ElMaestro 2020-12-11 07:59
      • Floating point arithmetic, again Ohlbe 2020-12-10 20:13
    • From bad to worse Ohlbe 2020-12-10 22:11
      • From bad to worse mittyri 2020-12-11 00:22
      • All is good Helmut 2020-12-11 00:36
        • Float is float PharmCat 2020-12-18 20:53
          • Float is float! Helmut 2020-12-20 23:27
            • rational solution in R mittyri 2020-12-21 13:49
              • related stuff Helmut 2021-01-14 12:53
                • related stuff SDavis 2021-02-09 12:02
                  • related stuff ElMaestro 2021-02-09 19:55