From bad to worse [Software]

posted by mittyri – Russia, 2020-12-10 23:22 (85 d 06:04 ago) – Posting: # 22136
Views: 1,762

Dear Ohlbe,

» If I get -2,77556E-17 when I type =(0.5-0.4-0.1), how comes that I get 0 when I type =(-0.1-0.4+0.5) ? The same figures, just in a different order ?

The order matters when you go to floating point arithmetic;-)


» If I get FALSE when I type =0=0.5-0.4-0.1, how comes that I get TRUE when I type =0=-0.4-0.1+0.5 ?

Rightyright. Why did it happen? The root cause is described here
Excel does do some additional operations under the hood.

» And to be fair to Excel, the same happens in [image]:
»
»

0==(0.5-0.4-0.1)
» [1] FALSE
» 0==(-0.4-0.1+0.5)
» [1] TRUE


» Are they first calculating -0.4-0.1=-0.5, which is nicely binarily coded, then -0.5+0.5=0 ?

Yes, take a look at this MS article
Both Excel and R are following IEEE 754, not surprising that the conditions are giving the same results. The problem is only in values representation to the user.

Kind regards,
Mittyri

Complete thread:

Activity
 Admin contact
21,370 posts in 4,463 threads, 1,495 registered users;
online 3 (0 registered, 3 guests [including 3 identified bots]).
Forum time: Saturday 05:26 UTC (Europe/Vienna)

If debugging is the process of removing bugs,
then programming must be the process of putting them in.    Edsger W. Dijkstra

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5