From bad to worse [Software]

posted by mittyri  – Russia, 2020-12-11 01:22 (1591 d 00:10 ago) – Posting: # 22136
Views: 18,662

Dear Ohlbe,

❝ If I get -2,77556E-17 when I type =(0.5-0.4-0.1), how comes that I get 0 when I type =(-0.1-0.4+0.5) ? The same figures, just in a different order?


The order matters when you go to floating point arithmetic. ;-)


❝ If I get FALSE when I type =0=0.5-0.4-0.1, how comes that I get TRUE when I type =0=-0.4-0.1+0.5?


Rightyright. Why did it happen? The root cause is described here
Excel does do some additional operations under the hood.

❝ And to be fair to Excel, the same happens in [image]:


0==(0.5-0.4-0.1)

[1] FALSE

0==(-0.4-0.1+0.5)

[1] TRUE


❝ Are they first calculating -0.4-0.1=-0.5, which is nicely binarily coded, then -0.5+0.5=0?


Yes, take a look at this MS article
Both Excel and R are following IEEE 754, not surprising that the conditions are giving the same results. The problem is only in values representation to the user.

Kind regards,
Mittyri

Complete thread:

UA Flag
Activity
 Admin contact
23,424 posts in 4,927 threads, 1,688 registered users;
25 visitors (0 registered, 25 guests [including 8 identified bots]).
Forum time: 02:32 CEST (Europe/Vienna)

It is true that many scientists are not philosophically minded
and have hitherto shown much skill and ingenuity
but little wisdom.    Max Born

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5