Floating point arithmetic, again [Software]

posted by ElMaestro  – Denmark, 2020-12-10 22:38 (1313 d 04:47 ago) – Posting: # 22130
Views: 9,932

Hi Ohlbe,


❝ Mmmm. But then if the problem comes from the nasty 0.1, how comes that if I type =(0.8-0.4-0.2-0.1-0.1) I get a nice 0, without any funky E-17 ? And then if I type =0=(0.8-0.4-0.2-0.1-0.1), or =0=0.8-0.4-0.2-0.1-0.1 I get a nice TRUE ?


Impossible to tell if this is an effect from Excel or from some degree of streamlining within the CPU.
It is fairly trivial (I think) at the CPU level as well as in Excel to check if you are handing objects of the type (x-x) and to substitute those with a zero where they occur.

Pass or fail!
ElMaestro

Complete thread:

UA Flag
Activity
 Admin contact
23,101 posts in 4,857 threads, 1,644 registered users;
79 visitors (0 registered, 79 guests [including 6 identified bots]).
Forum time: 04:26 CEST (Europe/Vienna)

One can show the following: given any rule, however “fundamental”
or “necessary” for science, there are always circumstances
when it is advisable not only to ignore the rule,
but to adopt its opposite.    Paul Feyerabend

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5