Floating Point Arithmetic

Eric Lippert has a serious of recent blog posts on Floating Point Arithmetic, if you were interested in or following any of my prior math quiz posts. 

Also, Eric Gunnerson talks about floating point arithmetic in a December 2003 post, and posts to a canonical discussion of floating point arithmetic in “What Every Computer Scientist Should Know About Floating-Point Arithmetic.”

Some more math trivia: True or false?

1. Math.Round(2.5) = 3.
2. Math.Round returns an int.
3. Both x and y are expressions of type Double and have the same exact bit representation of a Double in memory. Neither are special cases—ie, NaN, negative zero, denormalized numbers, or infinities. Therefore, x = y.
4. If x is a Double and x = x, then double.Parse(x.ToString()) = x.
5. Is MSDN documentation correct in stating the following about Double.Epsilon?
   
   “Two apparently equivalent floating point numbers might not compare equal because of differences in their least significant digits. For example, the C# expression, (double)1/3 == (double)0.33333, does not compare equal because the division operation on the left-hand side has maximum precision while the constant on the right-hand side is only precise to the visible digit.
   
   Instead, determine if the two sides of a comparison are close enough to equal for your purposes by comparing whether the absolute value of the difference between the left and right-hand sides is less than Epsilon.”

Comments

[Johan Ericsson]

1. False. Looks like Math.Round uses banker's rounding (at "5" round to the nearest even)
2. False (decimal or double). If it did return an "int" then you could overflow when giving very large "double" values.
3. ? seems like this would be true (but I'm obviously missing something)
4. False (I've found this process can drop some less significant digits.
5. This is weird. epsilon means something very different in C++ than it does in C#. In C++ epsilon is the smallest number that makes the following expression true:
1 + epsilon > 1

Epsilon, in C++, is normalized to 1. I'm not sure how you would use the one in C#.

And the documentation is completely misleading.

To compare if two values are close enough "for your purposes", you should compare against your own criteria for equality.

Obviously 1/3 != .33333. I'm surprised they have to spell that out.

And then to use their own "solution"
(double)1/3 - (double).33333 is so much greater than double.epsilon

[Scott Hanselman]

4, very false. I posted about this earlier this week. Intuitive to some, others not so much.

http://www.hanselman.com/blog/PermaLink,guid,bb01694d-d637-45fd-8e82-406dd3bc3027.aspx

[Stephen Gryphon]

On item #3, strictly speaking it is correct -- however the problem is x & y will not always have the same representation in memory.

In particular, x may be stored on the stack as a 64-bit float, whilst y may be stored in an internal register using an 80-bit format -- this is allowed according to the ECMA CLI Partition 1, section 12.1.3 for details.

So this means that even after what you think are equivalent operations / assignments you may actually end up with different representations in internal memory due to optimisation.

Of course, as soon as you try to debug or display that in-memory representation it pulls the value out of the internal register and does an implict conversion to a 64-bit float and you never see the comparison.

An example of this behaviour is found at:
http://www.yoda.arachsys.com/csharp/floatingpoint.html

Luckily the CLI specification (see end of section 12.1.3) provides a way around the problem -- if you do an explicit conversion operation, i.e. (double), then the value in the register must be converted and you will get the same in-memory represenation, resulting in equality.

- Sly