Binary representation [Software]

posted by ElMaestro  – Denmark, 2018-03-16 14:13 (2377 d 03:35 ago) – Posting: # 18561
Views: 7,103

Hi Mittyri,

❝ x=0.250000

❝ as it should be

:confused:


How nice, then it is also compiler-dependent...
The issue is that 0.05 cannot be represented accurately in a binary brain.
So, the compiler will need to deal with such a fraction and either round it up or round it down. Whatever it does, it can have downstream implications. In my case it implied that my simple naïve decimal logic (0.25 <=0.25) never evaluated to TRUE because it was not tested as such.
The solution could be to add e.g. 1e-8 to the righthand side. Dumb, but that's logic.
Computers suck.
:-D:-D:-D

Pass or fail!
ElMaestro

Complete thread:

UA Flag
Activity
 Admin contact
23,226 posts in 4,879 threads, 1,657 registered users;
35 visitors (0 registered, 35 guests [including 7 identified bots]).
Forum time: 18:48 CEST (Europe/Vienna)

The real purpose of the scientific method is to make sure
nature hasn’t misled you into thinking you know something
you actually don’t know.    Robert M. Pirsig

The Bioequivalence and Bioavailability Forum is hosted by
BEBAC Ing. Helmut Schütz
HTML5