A number of readers asked for an explanation of my offhand comment that Benford’s Law can be used to show that binary is the best base for doing floating point math. One of the desired properties of a floating point system is that the “representation error” is as small as possible. For example, suppose we…

# Tag: Floating Point Arithmetic

## Benford’s Law

While I was poking through my old numeric analysis textbooks to refresh my memory for this series on floating point arithmetic, I came across one of my favourite weird facts about math. A nonzero base-ten integer starts with some digit other than zero. You might naively expect that given a bunch of “random” numbers, you’d…

## Floating Point Arithmetic, Part One

A month ago I was discussing some of the issues in integer arithmetic, and I said that issues in floating point arithmetic were a good subject for another day. Over the weekend I got some questions from a reader about floating point arithmetic, so this seems like as good a time as any to address…

## Bankers’ Rounding

A number of people have pointed out to me over the years that VBScript’s Round function is a bit weird. It seems like it should be pretty straightforward — you pick the integer closest to the number you’ve got, end of story. But what about, say, 1.5? There are two closest integers. Do you go…

## Why does JScript have rounding errors?

Try this in JScript: window.alert(9.2 * 100.0); You might expect to get 920, but in fact you get 919.9999999999999. What the heck is going on here? Boy, have I ever heard this question a lot. Well, let me answer that question with another question. Suppose you did a simple division, say window.alert(1.0 / 3.0); Would…