| |
How do I get around the maximum int size? Displaying 1-5 of 5 total.
1
Technetium
|
I managed to get a solution to my previous problem for constructing parabolas. But now I have a new problem. Running in 640x480, this means that I will frequently get numbers greater than 600*600 in the equation y=ax^2 + b + c. That is still well below the limit, but the value of a needs to be stored in two values (numerator and denominator) in order to properly handle fractional values of a that are less than 1. However, this also means that the numerator for a could be very very large (with a denominator just slightly larger or nearly as large). When multiplying that by 600^2, the number easily surpasses the int limit and wraps around to a negative.
Suppose, for example, that "a" = 12800/14400, and I would therefore need to calculate:
(600*600*12800)/14400
... in order to get the ax^2 part of the quadratic equation. What kind of workaround is there so that I can get the right answer when all is done?
Posted on 2006-01-09 16:30:28
|
Technetium
|
Also, I know about the method of dividing by a large number and then multiplying at the end. There are two reasons I can't do this here:
1. In the end of the equation, I end up with numbers less than 1000. Although there are huge numbers, they get subtracted from other numbers which are almost the same size. If I divide by a large number, the difference between those two huge numbers will be cut off.
2. Depending on the circumstances, the variables could be much smaller, less than 100 each. Dividing by a large number would end up turning those values into 0s.
Posted on 2006-01-09 16:44:53
|
Overkill
|
The maximum integer size is 2 ^ 31 - 1 = 2,147,483,647.This is because it is a signed 32-bit integer, ie. 32 bit but it uses one bit to store if the number is negative or positive. If you overflow this limit, you'll end up with unexpected results that are likely in the negatives. The minimum limit for integer values is -2 ^ 31 = -2,147,483,648. I don't know of any easy way to work around this limit, but I'm sure it'd involve something like creating your own number storage system. :/
As for avoiding the division problems, you should use fixed point wherever possible and give the divided number to a seperate variable. That way you don't lose precision on the old results. Actually, I'm not really sure what you're trying to do.... er... So maybe cross that.
Also, can I ask why you're using parabolas? :D
Posted on 2006-01-09 17:01:36
|
Omni
|
BEFORE YOU MULTIPLY by 600^2.
I'm not sure of the formula you're using. But, basic principle.
When you must multiple by two huge numbers, without going out of bounds:
A * B = bad
Bitshift down then back up.
A>>2 * (B>>2) = C
return C<<4
This means accuracy takes a hit in the multiplication, but the product's integrity is, well...intact.
Or even better with less loss of data,
A>>2 * B = C
return C<<2
That allows for less accuracy loss.
I'm not quite sure how to apply this to your formula, but I believe this is what we need to do.
Posted on 2006-01-09 17:07:14
|
Omni
|
I'm not sure how your numerator or denominator system works. But,
(600*600*12800)/14400
Instead, try:
(600*600>>2 * 12800>>2)/(14400>>2) << 4
Posted on 2006-01-09 17:17:08
|
Displaying 1-5 of 5 total.
1
|
|
