From bad to worse [Software]

posted by mittyri  – Russia, 2020-12-11 01:22 (2012 d 01:21 ago) – Posting: # 22136
Views: 27,784

Dear Ohlbe,

❝ If I get -2,77556E-17 when I type =(0.5-0.4-0.1), how comes that I get 0 when I type =(-0.1-0.4+0.5) ? The same figures, just in a different order?


The order matters when you go to floating point arithmetic. ;-)


❝ If I get FALSE when I type =0=0.5-0.4-0.1, how comes that I get TRUE when I type =0=-0.4-0.1+0.5?


Rightyright. Why did it happen? The root cause is described here
Excel does do some additional operations under the hood.

❝ And to be fair to Excel, the same happens in [image]:

❝

❝

0==(0.5-0.4-0.1)

❝ [1] FALSE

❝ 0==(-0.4-0.1+0.5)

❝ [1] TRUE


❝ Are they first calculating -0.4-0.1=-0.5, which is nicely binarily coded, then -0.5+0.5=0?


Yes, take a look at this MS article
Both Excel and R are following IEEE 754, not surprising that the conditions are giving the same results. The problem is only in values representation to the user.
—
Kind regards,
Mittyri

Complete thread:

UA Flag
Activity
 Admin contact
23,653 posts in 4,991 threads, 1,570 registered users;
131 visitors (0 registered, 131 guests [including 45 identified bots]).
Forum time: 03:43 CEST (Europe/Vienna)

The idea is to try and give all the information to help others
to judge the value of your contribution;
not just the information that leads to judgment
in one particular direction or another.    Richard Feynman

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5