The DateTime struct represents dates as a 64 bit number that measures the number of “ticks” since a particular start date. Ten million ticks equals one second. That’s a quite high degree of precision. You can represent dates and times to sub-microsecond accuracy with a DateTime, which is typically more precision than you need. Not…

# Tag: precision

## Is there such a thing as too much precision?

Well, enough chit-chat, back to programming language design. Suppose you’re building electronic piano software. As we’ve discussed before, the “equal temperament” tuning for a piano goes like this: the 49th note from the left on a standard 88 key piano is A, and its frequency is 440 Hz. Each octave above or below that doubles…

## Fun with Floating Point Arithmetic, Part Five

I went to Joel Spolsky’s geek dinner at Crossroads the other night, which was a lot of fun. I didn’t get much of a chance to chat with Joel, as he was surrounded by a cadre of adoring fans three deep the whole time. I mostly hung out with KC and Larry and some other…

## Fun with Floating Point Arithmetic, Part Four

A reader also asked the other day why it is that in VBScript, CSng(0.1) = CDbl(0.1) is False. Forget about binary floating point for a moment. Suppose that we had two fixed-point decimal systems, say one with five digits after the decimal place and one with ten. You want to represent one-third. In our first…

## Floating Point And Benford’s Law, Part Two

A number of readers asked for an explanation of my offhand comment that Benford’s Law can be used to show that binary is the best base for doing floating point math. One of the desired properties of a floating point system is that the “representation error” is as small as possible. For example, suppose we…

## Eric’s Complete Guide To VT_DATE

I find software horology fascinating. The other day, Raymond said “The OLE automation date format is a floating point value, counting days since midnight 30 December 1899. Hours and minutes are represented as fractional days.” That’s correct, but actually it is a little bit weirder than that. I suspect that I may be the world’s…

## Why does JScript have rounding errors?

Try this in JScript: window.alert(9.2 * 100.0); You might expect to get 920, but in fact you get 919.9999999999999. What the heck is going on here? Boy, have I ever heard this question a lot. Well, let me answer that question with another question. Suppose you did a simple division, say window.alert(1.0 / 3.0); Would…