Floating Point calculations: comparing with zero


I was asked about floating point calculations. In particular, I can write code like this:


 


x=3.4


y=10


 


Result=x*y


 


?Result,Result=34


IF Result=34


      ?”This code will execute if result = 34″


ENDIF


 


*Output is 34.0 .T.


 


If you add 3 zeros to x (change x to 3.4000) then


*Output is 34.0000 .T.


 


Fox is keeping track internally of not only the floating point value of a number, but the number of decimal points of accuracy (I’ll call that NUMDPS). “3.4” has 1 implied decimal place of accuracy. “3.4000” has 4. When doing operations on a floating point number, the NUMDPS are tracked too. For example, when multiplying 2 floating point numbers, NUMDPS is the sum of the NUMDPS of the multiplicands. For addition, NUMDPS is the larger of the NUMDPS of the addends.


 


Floating point numbers are represented internally in binary. Run the BINTOC sample that ships with VFP to see the actual binary representation: Start VFP, choose Tools->Task Pane->Solution Samples->New in VFP9->BINTOC Binary Conversion


Type in 3.4 and choose “B” as the parameter to BINTOC.


 


It shows that 3.4 is represented internally as 33 33 33 33 33 33 08 40 hex, or 0011:0011 0011:0011 0011:0011 0011:0011 0011:0011 0011:0011 0000:1011 0100:0000  in binary


 


In fact, any multiple of 1/10 is an infinitely repeating fraction in binary. Just as 1/3 = .33333333333 in base 10, repeated fractions are not exact.


 


So what if you add 1/3 3 times?


 


.3333333


.3333333


.3333333


_______


.9999999 which is not exactly equal to 1.


 


So how is it that Fox says the result is equal to 34 ?


 


In general, to see if 2 values are equal, subtract one from the other, take the absolute value, and see if you get zero. This works fine for integers, but for real numbers, we may not get exactly zero.


 


We need then to determine how close to zero is close enough? Close enough means smaller than some number (often called epsilon) that is acceptable.


 


For acceptability we need to consider how many decimal places of accuracy are known.


 


If we get 34 – 33.9999999999 = 1E-10, and we only have 1 decimal digit of accuracy then we are clearly willing to accept that 1E-10 is indeed zero.


 


However, 34.000000000000 – 33.9999999999 = 1E-10 and we now have 10 digits of accuracy and it’s not so clear if this is close enough to zero.


 


Fox internally has an array of floats: 0.5E0, 0.5E-1, 0.5E-2, 0.5E-3, 0.5E-4, 0.5E-5, 0.5E-6, 0.5E-7, 0.5E-8 etc. NUMDPS is an index into this array to look up an appropriate epsilon to use.


 


56890


 

Comments (9)

  1. Calvin_Hsia says:

    I received this comment:

    You might want to caution your readers on the use of absolute epsilon values – actual precision in IEEE 764 is a function of the magnitude of the #, i.e. the discontinuities between numbers get larger as the numbers get larger – trying to use 1E-10 while operating on big numbers would not lead to happy camperdom 🙂

  2. Frédéric Steczycki (MVP, VFPEuropa beta tester) says:

    Hi Calvin,

    During the beta test, i pointed a little problem with some floatting point value and round() function. Maybe you’ve heard about, anyway, it is still present in release. I’d just like to have your POV on it.

    (all i’ve got, w/o any comment is "Bugged as 397968" <s>)

    LOCAL i,y,z

    CLEAR

    STRTOFILE("","c:test.txt",0)

    FOR i = 1 TO 100000

    y = i + .225000000000000

    z = i + .23

    IF ROUND(y,2)<>z THEN

    STRTOFILE(STR(i)+CHR(13)+CHR(10),"c:test.txt",1)

    ENDIF

    NEXT

    MODIFY FILE "c:test.txt"

    Regards,

    Fred

  3. Fabio Lunardon says:

    * EXPLAIN

    * you can look that the value stored is a approximation of I.225,

    * when the value is < I.225, the direct round fail

    LOCAL i,y,z

    CLEAR

    SET DECIMALS TO 0

    FOR i = 1 TO 4

    y = i + .2250000000000

    z = i + .23

    ?i,ROUND(Y,12),(y-I)*10000000000000,ROUND(y,2)<>z,ROUND(ROUND(y,12),2)<>z

    NEXT

    * then the solution is simple if you known the left zero position,

    * otherwise you have to found it !!!!!

    LOCAL i,y,z

    CLEAR

    STRTOFILE("","c:test.txt",0)

    FOR i = 1 TO 100000

    y = i + .225000000000000

    z = i + .23

    IF ROUND(ROUND(y,3),2)<>z THEN

    STRTOFILE(STR(i)+CHR(13)+CHR(10),"c:test.txt",1)

    ENDIF

    NEXT

    MODIFY FILE "c:test.txt"

  4. I received a question about how to use DEVMODE. Below is some sample code that retrieves the screen’s…

  5. Pisho says:

    Thank you for the information!