Ed Wilson: When 6 is 5.99999999999999

  • Article

One of the great things about being the Microsoft Scripting Guy, is answering the hundreds of e-mails a week that I receive. Yes, believe it or not, I try to answer all the e-mail that comes to the Scripter@Microsoft.Com email alias. Many times, the answer is, “dude, I am totally swamped, and would love to write your script for you, but I cannot. Why don’t you try the Scripting Guys Forum instead.” or words to that effect. If I am not very busy, I may very well go ahead and write the script for the person—after all they are my fans, and customers.

Occasionally, however, I receive an e-mail that causes me to stop what I am doing and investigate. This was the case yesterday, when I received an e-mail with this code in it.

a = 4

While a < 6

    Wscript.Echo a

    a = a + .1

Wend

It seemed innocent enough. Until I ran it! That is when I saw that the code would return the following information:

4
4.1
4.2
4.3
4.4
4.5
4.6
4.7
4.8
4.9
5
5.1
5.2
5.3
5.4
5.49999999999999
5.59999999999999
5.69999999999999
5.79999999999999
5.89999999999999
5.99999999999999

Cool, I thought! It turns out that the problem is with floating point numbers. Computers are binary, and they do not do fractions very well. In fact, they really do not do fractions at all. They convert everything to binary. So, some fractions work out perfectly, and others do not. There is no exact binary representation for 1/10 or 1/3 – the question then is how far out do you want to go before you have to do the rounding. If you are old enough, you may remember the so called “Pentium bug” from way back in 1994. Because of the floating point number issue, every time you add 1/10 you are actually adding the number nearest to 1/10 that can be represented in a 52 bit number. Eventually these errors add up as shown in our sample.

Interestingly enough, if had let the script run a little bit longer, we would have seen it “fix itself,” at least for a little while. For more information on this topic, take a look at Eric Lipperts Blog.

Technorati Tags: Ed Wilson,VBScript,Scripting Guy