From bad to worse [Software]

posted by mittyri – Russia, 2020-12-11 01:22 (781 d 06:55 ago) – Posting: # 22136
Views: 6,772

Dear Ohlbe,

❝ If I get -2,77556E-17 when I type =(0.5-0.4-0.1), how comes that I get 0 when I type =(-0.1-0.4+0.5) ? The same figures, just in a different order ?


The order matters when you go to floating point arithmetic;-)


❝ If I get FALSE when I type =0=0.5-0.4-0.1, how comes that I get TRUE when I type =0=-0.4-0.1+0.5 ?


Rightyright. Why did it happen? The root cause is described here
Excel does do some additional operations under the hood.

❝ And to be fair to Excel, the same happens in [image]:


0==(0.5-0.4-0.1)

[1] FALSE

0==(-0.4-0.1+0.5)

[1] TRUE


❝ Are they first calculating -0.4-0.1=-0.5, which is nicely binarily coded, then -0.5+0.5=0 ?


Yes, take a look at this MS article
Both Excel and R are following IEEE 754, not surprising that the conditions are giving the same results. The problem is only in values representation to the user.

Kind regards,
Mittyri

Complete thread:

UA Flag
Activity
 Admin contact
22,477 posts in 4,708 threads, 1,603 registered users;
9 visitors (0 registered, 9 guests [including 6 identified bots]).
Forum time: 08:17 CET (Europe/Vienna)

The mediocre teacher tells.
The good teacher explains.
The superior teacher demonstrates.
The great teacher inspires.    William Arthur Ward

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5