From bad to worse [Software]

posted by mittyri – Russia, 2020-12-10 23:22 (86 d 21:59 ago) – Posting: # 22136
Views: 1,778

Dear Ohlbe,

» If I get -2,77556E-17 when I type =(0.5-0.4-0.1), how comes that I get 0 when I type =(-0.1-0.4+0.5) ? The same figures, just in a different order ?

The order matters when you go to floating point arithmetic;-)


» If I get FALSE when I type =0=0.5-0.4-0.1, how comes that I get TRUE when I type =0=-0.4-0.1+0.5 ?

Rightyright. Why did it happen? The root cause is described here
Excel does do some additional operations under the hood.

» And to be fair to Excel, the same happens in [image]:
»
»

0==(0.5-0.4-0.1)
» [1] FALSE
» 0==(-0.4-0.1+0.5)
» [1] TRUE


» Are they first calculating -0.4-0.1=-0.5, which is nicely binarily coded, then -0.5+0.5=0 ?

Yes, take a look at this MS article
Both Excel and R are following IEEE 754, not surprising that the conditions are giving the same results. The problem is only in values representation to the user.

Kind regards,
Mittyri

Complete thread:

Activity
 Admin contact
21,371 posts in 4,463 threads, 1,496 registered users;
online 7 (0 registered, 7 guests [including 5 identified bots]).
Forum time: Sunday 21:21 UTC (Europe/Vienna)

When people learn no tools of judgment
and merely follow their hopes,
the seeds of political manipulation are sown.    Stephen Jay Gould

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5