Saturday 26 September 2009

Floating point precision ... I didn't think it worked like this

I thought floating point numbers were more accurate than this:
-0.48f + 0.02f = -0.46f .. right?
Nope. The answer is -0.459999979

Just some background. 0.2f is my drag factor to apply to the character's velocity, ie. gravity. The background/level is a set of horizontal strips. These are scrolled to the characters velocity each frame.

Reading around, ie. the result above should be precise to 6 decimal points. As I understand it, after 6 decimal points accuracy isn't precise.

This really screwed things up when scrolling the background, as the errors seemed to accumulate and I'd see seams. What I've got in place at the minute is to round to two decimal places, which gives me enough accuracy. Two other options are to use doubles or fixed point numbers (ie. ints).

If anyone's got a better idea. Let me know!

Glad I found a solution though, this has been bugging me for ages - this must trip a lot of people up if they have any type of sub-pixel scrolling.

1 comment:

Anonymous said...
This comment has been removed by a blog administrator.