|
 - Forum
 - iziBasic
Warning: A non-numeric value encountered in /web5/aldweb/www/aldweb_com/www/thread.php on line 257
 Sujet n° 1011 |
Problem with INT function |
le 09/12/2004 @ 23:11
par Chuck
|
Using the following code:
C=17.05*100
I$="Initial "+STR$(C,2)
I=MESSAGEBOX(I$,1)
C=INT(C)
I$="After INT "+STR$(C,2)
I=MESSAGEBOX(I$,1)
The first message shows the number 1705 as expected. The second message box shows the number 1704, incorectly.
I use this type of code to truncate numberic entries to two decimal places. Obviously, I have not shown the final step of dividing by 100 since it wasn't relevant to my problem. |
|
|
|
Warning: A non-numeric value encountered in /web5/aldweb/www/aldweb_com/www/thread.php on line 497
Réponse n° 1
--------
le 10/12/2004 @ 00:34
par aldweb
visiteur
|
Ok, Mister "Number One Bugs Finder"  I will give a look to this function too.
It's really great to have someone like you for finding things like this one and all the previous ones you found in the past 
Thank you.
@+
aldweb |
|
|
Warning: A non-numeric value encountered in /web5/aldweb/www/aldweb_com/www/thread.php on line 497
Réponse n° 2
--------
le 11/12/2004 @ 14:42
par aldweb
visiteur
|
Chuck,
I have investigated, and the bug does not come from iziBasic itself. I have simulated the exact same source code in the compiler I use for iziBasic... and I got the same error.
Indeed C holds 1.7049999e03 as a value after you assigned it 17.05x100.
Then, when you ask for INT of this value, it is right to return 1704, not 1705.
And, STR$ returns with the right precision because it truncates the value in an intelligent manner (meaning if looks for 17.049 => 17.05).
I will get in touch with Philippe, PP compiler's developer to investigate more on this topic.
Cheers,
@+
aldweb |
|
|
|
Warning: A non-numeric value encountered in /web5/aldweb/www/aldweb_com/www/thread.php on line 497
Réponse n° 3
--------
le 11/12/2004 @ 21:40
par Pierre
visiteur
|
Hi Chuk !
As I am indirectly involved into the IziBasic project (I initially developped a free simple math library dramatically improved by Philippe Guillot, wich is used by Aldweb into IziBasic), I'll try to give you a simple explanation :
IziBasic uses internally a 32 bits real library. So, these reals are 4 bytes coded (just as 'integer' within standart Pascal). Internal rounding operations are made according to IEEE 754 (see http://stevehollasch.com/cgindex/coding/ieeefloat.html ).
You will never get <no error> results with a such real library, even if you increase the bytes coding structure. Of course, errors are more evident with a 32 bits lib than with a 64 one as MathLib.
Results of your example is returning are inherent to 32 bits "IEEE real" :
0x44D51999 -> 1704.799927
0x44D52000 -> 1705.
0x44D52001 -> 1705.000122
As each number is internally 'rounded', operations are adding errors ...
One bit more into internal real representation will return a -0.200073000 error as real in this interval 
The only way to improve accuracy is to use MathLib or BCD representation. The first option is fast and easy, the second is slow and to be coded as, at the moment, there is no such PalmOs library developped.
I hope these explanations will give you a begining of explanations.
... Aldweb, j'espère que j'ai été assez clair  : je reviendrai plus souvent sur ce forum !
Pierre
|
|
|
|
Warning: A non-numeric value encountered in /web5/aldweb/www/aldweb_com/www/thread.php on line 497
Réponse n° 4
--------
le 12/12/2004 @ 16:35
par Philippe Guillot
visiteur
|
Computers mainly work with radix 2 representation for numbers. This implies that there exists rounding error when converting from radix 10 to radix 2. For instance, the fraction 1/5 is 0.2 in radix 10 and is 0.001100110011... in radix 2, with an infinite sequence of 0011. Thus there always exist a rounding error when converting to a computer representation.
Increasing the size does not supress the problem. One must be convinced that computation with real numbers are often approximation and results are false, although they are acceptable for most application. If you need exact values, you must work wigh integers.
|
|
|
|
Warning: A non-numeric value encountered in /web5/aldweb/www/aldweb_com/www/thread.php on line 497
Réponse n° 5
--------
le 12/12/2004 @ 20:25
par Chuck
visiteur
|
Thank you all for the explanations. I understand what is happening now.
Will using the ROUND function after multiplying by 100 solve my issue?
If not, I can always parse the string version and truncate that way. |
|
|
|
Warning: A non-numeric value encountered in /web5/aldweb/www/aldweb_com/www/thread.php on line 497
Réponse n° 6
--------
le 12/12/2004 @ 21:15
par aldweb
visiteur
|
Hi Chuck,
Did you see how Pierre and Philippe rule?
Yeah, these guys really rule!
With ROUND, any value < x.5 will be rounded to x and any value >= x.5 will be rounded to x+1.
In other words, it is funny to see that ROUND(X) and INT(X+0.5) are the same thing
To make sure you got the right result, if you know that you always work with 2 digits precision, you could maybe attempt something like that :
C=17.05*100
C=C+0.001 '3 digits precision factor
C=INT(C)
But, again, give it a try. I am not sure this will be OK in all cases...
Cheers,
@+
aldweb |
|
|
|
sujet actif
 sujet clos
 Important!
 Nouveau message -
 Rectifier message
 Clôturer sujet
 Remonter
|
|
I. aldweb
![[de]](img/de.gif)
![[es]](img/es.gif)
![[it]](img/it.gif)
![[pt]](img/pt.gif)
visiteurs
Haut 
© 2004-2013 
