Floating point arithmetic, again [Software]

posted by Ohlbe – France, 2020-12-10 21:46 (655 d 17:14 ago) – Posting: # 22131
Views: 6,506

Hi ElMaestro,

» Impossible to tell if this is an effect from Excel or from some degree of streamlining within the CPU.

Same behaviour in [image]:

0==(0.8-0.4-0.2-0.1-0.1)
[1] TRUE


Regards
Ohlbe

Complete thread:

UA Flag
Activity
 Admin contact
22,391 posts in 4,685 threads, 1,594 registered users;
online 7 (0 registered, 7 guests [including 5 identified bots]).
Forum time: Tuesday 16:00 CEST (Europe/Vienna)

Statistics. A sort of elementary form of mathematics which consists of
adding things together and occasionally squaring them.    Stephen Senn

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5