
 Forum  iziBasic
Warning: A nonnumeric value encountered in /web5/aldweb/www/aldweb_com/www/thread.php on line 257
Topic # 1011 
Problem with INT function 
09/12/2004 @ 23:11 by Chuck

Using the following code: C=17.05*100 I$="Initial "+STR$(C,2) I=MESSAGEBOX(I$,1) C=INT(C) I$="After INT "+STR$(C,2) I=MESSAGEBOX(I$,1)
The first message shows the number 1705 as expected. The second message box shows the number 1704, incorectly.
I use this type of code to truncate numberic entries to two decimal places. Obviously, I have not shown the final step of dividing by 100 since it wasn't relevant to my problem. 


Warning: A nonnumeric value encountered in /web5/aldweb/www/aldweb_com/www/thread.php on line 497
Answer n° 1  10/12/2004 @ 00:34 by aldweb
visitor 
Ok, Mister "Number One Bugs Finder" I will give a look to this function too. It's really great to have someone like you for finding things like this one and all the previous ones you found in the past
Thank you.
@+ aldweb 


Warning: A nonnumeric value encountered in /web5/aldweb/www/aldweb_com/www/thread.php on line 497
Answer n° 2  11/12/2004 @ 14:42 by aldweb
visitor 
Chuck,
I have investigated, and the bug does not come from iziBasic itself. I have simulated the exact same source code in the compiler I use for iziBasic... and I got the same error. Indeed C holds 1.7049999e03 as a value after you assigned it 17.05x100. Then, when you ask for INT of this value, it is right to return 1704, not 1705. And, STR$ returns with the right precision because it truncates the value in an intelligent manner (meaning if looks for 17.049 => 17.05).
I will get in touch with Philippe, PP compiler's developer to investigate more on this topic.
Cheers,
@+ aldweb 


Warning: A nonnumeric value encountered in /web5/aldweb/www/aldweb_com/www/thread.php on line 497
Answer n° 3  11/12/2004 @ 21:40 by Pierre
visitor 
Hi Chuk !
As I am indirectly involved into the IziBasic project (I initially developped a free simple math library dramatically improved by Philippe Guillot, wich is used by Aldweb into IziBasic), I'll try to give you a simple explanation :
IziBasic uses internally a 32 bits real library. So, these reals are 4 bytes coded (just as 'integer' within standart Pascal). Internal rounding operations are made according to IEEE 754 (see http://stevehollasch.com/cgindex/coding/ieeefloat.html ).
You will never get <no error> results with a such real library, even if you increase the bytes coding structure. Of course, errors are more evident with a 32 bits lib than with a 64 one as MathLib.
Results of your example is returning are inherent to 32 bits "IEEE real" :
0x44D51999 > 1704.799927 0x44D52000 > 1705. 0x44D52001 > 1705.000122
As each number is internally 'rounded', operations are adding errors ...
One bit more into internal real representation will return a 0.200073000 error as real in this interval
The only way to improve accuracy is to use MathLib or BCD representation. The first option is fast and easy, the second is slow and to be coded as, at the moment, there is no such PalmOs library developped.
I hope these explanations will give you a begining of explanations.
... Aldweb, j'espère que j'ai été assez clair : je reviendrai plus souvent sur ce forum !
Pierre



Warning: A nonnumeric value encountered in /web5/aldweb/www/aldweb_com/www/thread.php on line 497
Answer n° 4  12/12/2004 @ 16:35 by Philippe Guillot
visitor 
Computers mainly work with radix 2 representation for numbers. This implies that there exists rounding error when converting from radix 10 to radix 2. For instance, the fraction 1/5 is 0.2 in radix 10 and is 0.001100110011... in radix 2, with an infinite sequence of 0011. Thus there always exist a rounding error when converting to a computer representation. Increasing the size does not supress the problem. One must be convinced that computation with real numbers are often approximation and results are false, although they are acceptable for most application. If you need exact values, you must work wigh integers.



Warning: A nonnumeric value encountered in /web5/aldweb/www/aldweb_com/www/thread.php on line 497
Answer n° 5  12/12/2004 @ 20:25 by Chuck
visitor 
Thank you all for the explanations. I understand what is happening now.
Will using the ROUND function after multiplying by 100 solve my issue?
If not, I can always parse the string version and truncate that way. 


Warning: A nonnumeric value encountered in /web5/aldweb/www/aldweb_com/www/thread.php on line 497
Answer n° 6  12/12/2004 @ 21:15 by aldweb
visitor 
Hi Chuck,
Did you see how Pierre and Philippe rule? Yeah, these guys really rule!
With ROUND, any value < x.5 will be rounded to x and any value >= x.5 will be rounded to x+1. In other words, it is funny to see that ROUND(X) and INT(X+0.5) are the same thing
To make sure you got the right result, if you know that you always work with 2 digits precision, you could maybe attempt something like that : C=17.05*100 C=C+0.001 '3 digits precision factor C=INT(C) But, again, give it a try. I am not sure this will be OK in all cases...
Cheers,
@+ aldweb 


topic active
topic closed
Sticky
New message 
Correct message
Close topic
Make sticky

