Reals and Integers, Apples and Oranges
Do you remember when you first learned about real numbers? I do. Things had been so easy when all I had to track was whole numbers. Even division was nice and neat. If you couldn't divide evenly you just moved the remainder of to the side. Nice and smooth it was. But they wouldn't let that last. They had to introduce those digits to the right of the decimal point. And those repeating fractions! 1/3 for example. A row of threes that never ended. When I started programming I found out the hard way, as most of us do, that 1/10 was a repeating decimal in binary. Just what I needed - more repeating fractions.
This all came back to me recently when I received an email from a high school computer science teacher. Her class was using the Integer Division operator in Visual Basic and were seeing some results they didn't expect. Now as far as I know most programming languages have only one division operator (/). Visual Basic being a more powerful and advanced language than some others (that's my story and I'm sticking to it) has two division operators. The regular operator (/) returns a floating point quotient which includes any remainder. The Integer Division operator (\) returns the integer quotient, which drops the remainder. More on this in a moment.
The teacher was seeing results that suggested that the Integer Division was rounding the quotient off rather than just dropping the remainder completely. What was actually happening is a little more complicated. The Integer Division operator divides integers and only integers. If there are non-integers involved they are first converted to integers. And this is what caused some confusion.
When the numbers are converted to integers they are subject to something called banker's rounding. This is in accordance with IEEE Standard 754, section 4 so it is not some arbitrary activity. What this means is that some rounding up or down may happen before the division is calculated and that may give results different from doing the division using floating-point numbers and truncating the result.
Now of course with either regular or integer division you can have the result put in an integer variable. In the case of the regular division operator the result will be converted to an integer and may be rounded up. You may want to remember that as well. There is a truncate function in Visual Basic by the way if you really want that to happen.
Let's look at an example of what can happen.
5.6 / 2.4 is 2.333333... (I can't seem to get away from those repeating fractions)
Rounding that off will give you an integer answer of 2 because 2.333333 rounds down. But 5.6 \ 2.4 is different. In that case the 5.6 is rounded up to 6. The 2.4 is rounded down to 2. Both are done before the division takes place so what is actually divided is 6 by 2 which of course is three. Getting the slash in the right direction can be important.
I think this is a good reminder of several things. One is that the order in which one does something is almost always important. The other is that even things that appear to be simple can sometimes have complications that are not obvious. It also pays to read the documentation.
I've long said that "reading the documentation is the shortcut" and I think this example helps make the case.
| Share this post : |  |
 |
 |
 |
 |
 |
 |