Small Basic – Mystery of Round Operation

Computer calculation always has some errors.  I wrote following program to investigate how Math.Round operation works.  Guess what type of errors occur in following case.

TextWindow.WriteLine(Math.Round("2.50000000000000017763568394"))    ' 2
TextWindow.WriteLine(Math.Round( 2.5000000000000001776356839401 ))  ' 2
TextWindow.WriteLine(Math.Round("2.500000000000000200000"))         ' 2
TextWindow.WriteLine(Math.Round("2.5000000000000002000"))           ' 2
TextWindow.WriteLine(Math.Round("2.500000000000000200"))            ' 2
TextWindow.WriteLine(Math.Round("2.5000000000000002"))              ' 2
TextWindow.WriteLine(Math.Round( 2.5000000000000051 ))              ' 2
TextWindow.WriteLine(Math.Round( 2.5000000000000052 ))              ' 3
TextWindow.WriteLine(Math.Round("2.5000000000000001776356839401"))  ' 3
TextWindow.WriteLine(Math.Round("2.50000000000000020"))             ' 3

Actuary, I can't understand yet what happens in the program above...

Comments (3)

  1. Math.Round() Option Rounds A Decimal To The Nearest Even Number. For Example
    If You Try
    TextWindow.WriteLine(Math.Round(“11.5”)) ‘You Will Get The Output As 12
    TextWindow.WriteLine(Math.Round(“12.5”)) ‘You Will Get The Output As 12
    This Is Because The Math.Round() Rounds The Decimal To The Nearest Even Number

    1. RoshanPriyaKumar, thank you for your comment. Yes, that the most important point to understand Math.Round() operation.

  2. Tryhest says:

    it has to do also with some issues in .net frmwrk:

    however the numbers are not stored with same decimal precision, i.e. the larger the number is, the lower decimals are kept:
    example: GVZ923
    as seen, total number of significant digits is 16 in sb math

Skip to main content