Close I. aldweb

Close II. PC Freeware

Close III. Palm Freeware

Close IV. Palm Shareware

Close V. iziBasic Palm

Close VI. Palm Knowledge

Close VII. Pocket Computer

Close VIII. miniPortail


To receive news about this website, consider subscribing to our Newsletter.
298 Subscribers
Webmaster - Infos


   visitors online

article.gifVI. Palm Knowledge - 3. Development Tools Benchmark


Version 12 - July 25th, 2009
Made by Laurent Duveau
web Site :
e-mail :


Version 12 (July 25th, 2009)
  • Added FastBasic
Version 11 (October 17th, 2006)
  • Added NaPP ("private preview release")
Version 10 (January 7th, 2006)
  • Added PocketC Architect (thanks to Christopher Hill)
  • Added Palm Basic
Version 9 (February 23rd, 2005)
  • Added Quartus Forth (thanks to Marc Furrer)
  • Updated iziBasic results in its release #5 (benchmark runs faster)
  • Grouped the tools in 4 categories in the table of results
Version 8 (October 22nd, 2004)
  • Added AppForge (thanks to Ingbert Grimpe)
  • Changed type (from interpreted to runtime) for HotPaw Basic and SmallBasic as it appears that these 2 tools use an internal bytecode
Version 7 (September 5th, 2004)
  • Updated iziBasic results in its release #2 (benchmark runs faster)
Version 6 (July 22nd, 2004)
  • Changed the referent device: from Sony Clié N770C to Palm Tungsten C, so I had to run all timings again!
  • As a consequence, I started to include ARMlets (Palm OS 5) versions of Bench2
  • Updated many developement tools to their latest version: cbasPad5, HotPaw Basic, HSPascal, LyME, OnBoardC, picoBASIC, PLua, PP, Rexx
  • Added LaFac HELP, µBAS, µPAS et µC (my own multi-mini-languages developement tool)
  • Added iziBasic
Version 5 (February 17th, 2004)
  • Added LispMe
  • Added CASL
Version 4 (September 18th, 2003)
  • Added OrbForms Designer
  • Updated PocketC in its latest version
  • Updated PP in its latest version
Version 3 (September 1st, 2003)
  • Added HB++, brand new Basic compiler and RAD development tool
Version 2 (December 12th, 2002)
  • Complete rewrite of the Bench2 routine to better benchmark the development tools.
    Thanks to all of the people who provided me with sharp analysis, bright comments and good ideas.
  • As a consequence, I highly updated this web page!
  • Refreshed the results for NSBasic/Palm in its new version 3.0.0d
  • Refreshed the results for PP in its new version 1.05a
  • Added results for the cbasPad, Jump2, Rexx and Satellite Forms development tools
Version 1 (October 22nd, 2002)
  • Initial release


I developed the Bench2 benchmark routine with three goals in mind that are ordered by level of importance:
  1. Provide a simple but quite rich example of a routine in different languages, for educational purposes. As a result, the comparizon of the different source codes provided on this page is very interesting.
  2. Find a routine that an average amateur programmer could possibly write, not especially optimized but giving the same right result. Result is more important than means...
  3. In order to give a playful purpose to the writing of the same routine in different languages, try to evaluate compilers & interpreters efficiency, designed to work on the Palm OS environment where performance and speed are of key importance.
Michael Winikoff already did a very impressive job on this topic on his PalmPilot Software Development - Alternatives to C page by benchmarking the different development tools with a very simple benchmark routine (that I nickname Bench1 and this is why my own routine is named Bench2):

program start
start timer
x = 1000
y = 0
while (x>0) do
x = x-1
y = y+2
end do
stop timer
display execution time
program end

This is a very good idea and initiative, but this benchmark is fairly limited as it only tests for single decrementations and incrementations loop on integer numbers. Furthermore, an intelligent compiler will easily find out that the x and y variables are not used afterwards so it will just skip the execution of the loop (this is the case with the newest version 8 of CodeWarrior... a Delphi compilation would give a warning message and let the programmer decide for the action to take).

So, I felt like refining this benchmark analysis and I tried to extend the benchmark to floating point and integer numbers, tests (if), loops (for), with the following benchmark routine which is a Perfect Numbers finder routine for the first 500 or 1000 integer numbers:

program start
for B=1 to 2 step 1 do
execute PerfectNum routine
end for
program end

PerfectNum routine
routine start
start timer
P = 0
for N=2 to B*500 step 1 do
M = N div 2
S = 1
for D=2 to M step 1 do
if B=1 then if (N/D)=(N div D) then S = S+D
if B=2 then if (N modulo D)=0 then S = S+D
end for
if S=N then P = P+1
end for
stop timer
if P<>3 then execution time = 0
display execution time
routine end

Note : "div" stands for "integer division"
then 3 div 2 = 1 and 3/2 = 1.5

The first 4 perfect numbers are: 6, 28, 496 and 8128. Since we look for perfect numbers between 2 and 500 or 1000, P should hold 3 at the end of the benchmark routine. If this were not the case, a time of 0 second will be returned, thus giving the code error signification. Furthermore, this reuse of the P variable makes sure that an intelligent compiler will not detect that this variable is never used after it has been provided with a value.

Additional ideas were:
  • Keep the benchmark program as simple as possible to allow a real comparizon between the different languages.
  • Keep it as similar as possible, try to avoid optimizations like inc(P) instead of P:=P+1 in Pascal or S+=D instead of S=S+D in C. I do not know if this has an impact on performance, but I think that it is easier to compare the routine in the different language this way.
  • Floating point and integer numbers management in Palm OS is a tricky subject and it brought me a lot of e-mails and comments when I released the initial Bench2 routine that was not doing the modulo test. The DragonBall processor has a native management for 16 bit integers on which some development tools rely when others have developped a 32 bit integers management. So, of course, the second ones can be a little bit slowlier in execution (but not always, for instance PocketStudio runs just as fast using the integer type or the longint type). Most tools rely on the Palm OS APIs for managing float numbers and the same remark as for the integer numbers is valid for simple float numbers or double float numbers. I finally decided to work with the default integer and floating point types provided by the different development tools for the Bench2 test (but, for example, OnBoardC runs faster with the Double type than with the Float type). This is justified by the fact that we are taking into account our second objective (a routine that an average amateur programmer could possibly write, not especially optimized but giving the same right result) with particular attention.
  • The loop [ if (N modulo D)=0 then S = S+D ] (work on integer numbers - this is the add-on analysis in Bench2 version 2) runs for as much as four times, 249001 times, than the one with [ if (N/D)=(N div D) then S = S+D ] (work on float numbers - this was the only analysis made in Bench2 version 1) which runs 62001 times. This is to take into account comments and feedbacks that came to me after I had posted version 1 of Bench2. Indeed, I was nicely told and I acknoledge that benchmark on floating point number is not always significative. Therefore, Bench2's total benchmark time will be level-headed with this different number of loops execution.


All times given here are those found on my Palm Tungsten C (400 MHz XScale processor) when running Bench2, either stand-alone PRC program (for compilers), using a runtime (for pseudo-coded languages) or within the interpreter's IDE.

  • the tools running in 68k mode (PACE) are in Blue, those running in native ARM mode are in Red.
  • to ease the reading of the results, I have chosen to group the tools into 4 categories:
    1. less than 10 secondes
    2. between 10 and 100 secondes
    3. between 100 and 1000 secondes
    4. more than 1000 secondes

Dev. Tool Version Language Type Time 1 Time 2 Total Time
NaPP preview Pascal compiled 0.11 sec 0.06 sec 0.17 sec
PP 2.08q Pascal compiled 0.14 sec 0.07 sec 0.21 sec
FastBasic 1.2 Basic compiled 0.86 sec 0.09 sec 0.95 sec
CodeWarrior 8.0 C compiled 0.70 sec 0.49 sec 1.19 sec
HSPascal 2.1.0 Pascal compiled 0.82 sec 0.52 sec 1.34 sec
HB++ 1.02 Basic compiled 1.12 sec 0.63 sec 1.75 sec
PP 2.08q Pascal compiled 0.71 sec 1.06 sec 1.77 sec
PocketStudio 1.1 Pascal compiled 1.08 sec 0.89 sec 1.97 sec
OnBoardC 2.4.2 C compiled 1.65 sec 0.58 sec 2.23 sec
Jump2 2.0ß8 Java compiled 1.33 sec 1.26 sec 2.59 sec
FastBasic 1.2 Basic compiled 1.55 sec 1.65 sec 3.21 sec
Quartus Forth 1.2.1 Forth compiled 2.89 sec 1.56 sec 4.45 sec
cbasPad 5.3b0i Basic interpreted 1.07 sec 3.60 sec 4.67 sec / GCC 2.5 C compiled 5.34 sec 0.51 sec 5.85 sec
SuperWaba 4.21b Java runtime 3.06 sec 6.31 sec 9.37 sec
AppForge 5.1.1 Basic runtime 12.92 sec 10.01 sec 22.93 sec
CASLpro / GCC 4.1.3 CASL compiled 12.46 sec 18.38 sec 30.84 sec
FastBasic 1.2 Basic interpreted 10.68 sec 37.53 sec 48.21 sec
Sun MIDP (*) 1.0 Java runtime 12.20 sec 42.10 sec 54.30 sec
PLua 1.0 Lua runtime ~12 sec ~50 sec ~62 sec
OrbForms 2.1.1 C runtime 16.14 sec 49.20 sec 65.34 sec
PocketC Architect 4.0.0 C runtime 16.22 sec 49.29 sec 65.51 sec
LaFac - HELP 1.1 HELP runtime 18.73 sec 63.69 sec 82.42 sec
iziBasic 5.0 Basic runtime 21.61 sec 75.80 sec 97.41 sec
HotPaw Basic 1.4.1 Basic runtime 29.54 sec 108.50 sec 138.04 sec
SmallBASIC 0.8.2 Basic runtime 45.14 sec 133.27 sec 178.41 sec
LispMe 3.21 Scheme runtime 39.76 sec 140.60 sec 180.36 sec
CASL 4.1.3 CASL runtime 36.16 sec 147.24 sec 184.40 sec
NS Basic/Palm 3.0.0d Basic runtime 44.71 sec 155.11 sec 199.82 sec
LaFac - µBAS 1.1 Basic runtime 47.31 sec 174.29 sec 221.60 sec
LaFac - µC 1.1 C runtime 64.90 sec 245.17 sec 310.07 sec
LaFac - µPAS 1.1 Pascal runtime 64.91 sec 245.20 sec 310.11 sec
PocketC 6.0.0 C runtime 82.51 sec 265.89 sec 348.40 sec
REXX 1.00 Rexx interpreted ~89 sec ~265 sec ~354 sec
LyME 2.8 LME interpreted 124.16 sec 577.78 sec 701.94 sec
Palm Tcl 0.4 Tcl interpreted 203.64 sec 607.21 sec 810.85 sec
Palm Basic (**) 1.02 Basic interpreted ~280 sec ~1152 sec ~1432 sec
picoBASIC (*) 1.01 Basic interpreted ~4590 sec ~15931 sec ~20521 sec
Satellite Forms 5.0.0 Basic runtime [does not run in ARM devices]
cbasPad 0.97b4 Basic interpreted [does not run in ARM devices]

And the same results shown in a graphical way :

Dev. Tool Bar Chart (shorter is better)
Quartus Forth
cbasPad / GCC
Sun MIDP (*)
PocketC Archit.
LaFac - HELP
HotPaw Basic
NS Basic/Palm
LaFac - µBAS
LaFac - µC
LaFac - µPAS
Palm Tcl
Palm Basic (**)
picoBASIC (*)

(*) : picoBASIC and Sun MIDP only work with integer numbers and they do not have a float numbers management. This is why loop [ if B=1 then if (N/D)=(N div D) then S = S+D ] cannot be run as is. Replacing it with [ if B=1 then if (N-(N/D)*D)=0 then S = S+D ] gives a a way of not removing these two development tools from the Bench2 comparizon even if they cannot then be compared exactly with the others.
(**) : Palm Basic does not have a modulo operator. This is why loop [ if B=2 then if (N modulo D)=0 then S = S+D ] cannot be run as is. Replacing it with [ if B=2 then if (N-(N div D)*D)=0 then S = S+D ] provides with an equivalent formula.


As you could see when reading the previous charts, the results are different depending on what you consider. None of the 3 analysis (overall Bench2 with 1st and 2nd loops, 1st loop only or 2nd loop only) gives the same podium.

We easily get to see a huge gap between the very first timing (PP in ARM mode) and the very last one (picoBASIC Integer in 68k mode) : 0.21 seconde to be compared to 20521 secondes. This is a gap of more than 5 hours 42 minutes when the first one runs in less that a quarter of a second!!!
If we take into account the second timing (CodeWarrior in 68k mode at 1.19 secondes) and the last "reasonable" one (Palm Tcl in 68k mode at 810.85 secondes), thus considering that the extremes results are not significative, we still find ourselves with a huge gap of almost 13 minutes and a half to be compared with 1.19 secondes !!!

Without any surprise, we see that compiled languages are the fastest ones, ahead of the runtime ones, themselves performing better than the interpreted ones.

I am personally quite happy to see that my own HELP, µBAS, µPAS and µC languages which are included in LaFac are in the average (self-satisfaction !).
I am even very happy to see that iziBasic is the best among the well-known Basic tools, but with less GUI functionalities.

So, where is the truth? By the way, is there a truth in this exercise?

Let's go back to our initial objectives and remember this one: find a routine that an average amateur programmer could possibly write, not especially optimized but giving the same right result. Result is more important than means....

Taking into account this objective, I let you establish by yourself your very personal podium!

As a conclusion, let's not forget, as George Henne from NSBasic Corporation reminded me in a very right analysis, that a benchmark like Bench2 needs to be taken in context of a total development project in which the developer, either professional or amateur, should answer to the following questions (that I would gently call the " 7 HOW questions ") for choosing a development tool:
  • How much of the job is CPU bound?
  • How long will it take to develop my app?
  • How good is the look and feel of the resulting project?
  • How compatible will it be with various Palm OS devices?
  • How is the support?
  • How easy is it to deploy?
  • How long has the tools vendor been around? Is he likely to be there for the full life cycle of my product?

If you answer to this questions, you will find your truth. And they are many truths since we all are so different people.
Let's also realize that we are very lucky to have such a choice of developement tools for our Palm devices!


Only the final results have been put in this page. For the detail of the source codes, of the links to the development tools and the single results for all these tools, please refer to the Bench 2 source codes page.

By the way, it is a very instructive exercise for the amateur developer as well as for the professional one, to read the source codes in the different languages. Therefore, I highly invite you to read them.

If you would like to help me extend this list of already benchmarked development tools and are working on one which is not listed here yet or in a newer version, I would be more than happy if you get in touch with me.

Creation date : 01/01/2003 @ 00:00
Last update : 30/07/2009 @ 21:05
Category : VI. Palm Knowledge

Print the article Print the article

react.gifReactions to this article

Reaction #18 

by aldweb 25/02/2005 @ 00:33


Relis bien la conclusion, surtout le paragraphe tout droit sorti d'un échange d'e-mails avec George Henne, échange très similaire à cette discussion que tu as engagée ici.
Je crois bien que nous sommes fondamentalement d'accords !

Je suis bien d'accord aussi sur la finalité des applications Palm générales.
Mon benchmark Speedy qui est dans le TOP 50 des applis les plus utilisées selon fonctionne sous un tout autre principe que ce petit exercice de style Bench2, avec bien cet objectif de donner une estimation de rapidité d'une application "moyenne", car pas 2 applis se ressemblent néanmoins.

Reaction #17 

by Miellaby 17/02/2005 @ 11:22

computionnels: lire "computationnels"

Reaction #16 

by miellaby 17/02/2005 @ 11:21

Le calcul numérique que tu as choisis permet de comparer des langages procéduraux entre eux (C, Pascal, Java...) ou de comparer les basics entre eux.

Mais il ne permet pas de comparer le tout à des langages haut-niveau. Ces derniers sont conçus pour gérer de l'information, pas pour faire des calculs computionnels primaires.

Il s'agit de savoir à quoi va servir le langage. Basic est bien pour faire un petit convertisseur d'Euro en 3 lignes, Pascal est bien pour programmer un logiciel rapide mais pas trop complexe (un jeu d'action).

Mais voilà. Les logiciels pour lesquels les Palms sont prévus au départ, c'est avant tout des gestionnaires... de rendez-vous, de n° de téléphone, de scoubidous... avec des possibilités de classement, de tri, de recherche, de renvoi et toutes ces sortes de chose.

Or des langages scriptiques lents et compliqués quand il s'agit de faire des calculs (en Tcl, il faut exécuter une commande pour faire une addition), se révèlent faciles à employer et très efficaces (hashTable à tous les étages) pour ce genre d'application.

Ils vont même être plus rapides à l'exécution que du Pascal compilé quand la quantité de données augmente, à moins d'avoir mis au point des hashTables de hashTables en Pascal, ce qui n'est à la portée de presque personne.

je vais être plus précis avec l'exemple du gestionnaire d'adresse. Voilà un début de programme:

En basic --------------------
DIM N$­­(10) 'Les noms
DIM A$­­(10) 'Les adresses
N = 0 'Le nombre de nom

'A transformer en procedure
N = N + 1
N$­­(N) = "sylvain"
A$­­(N) = "palmville"

'Trouver une adresse
FOR I = 1 TO N
IF N$­­(I) = C$­­ THEN PRINT A$­­(I)

En Tcl (par exemple) --------------------
# a : la table des adresses, indexé par les noms
set a(sylvain) "palmville"
set c [gets]
puts $­­a($­­c)

... à partir de là, toutes les opérations dont j'ai parlé plus haut (recherche, tri...) se font en quelques instructions, alors qu'en Basic, ca deviendrait vite l'usine à gaz.

Reaction #15 

by aldweb 11/02/2005 @ 15:10


Merci pour tes bien sympathiques compliments.

Il n'y a pas de problème que tu interviennes. Au contraire !
Si tu as cru que j'étais vexé, point du tout.
Tu ouvres un débat, je trouve tes remarques intéressantes, donc je me permets à mon tour de te répondre

Pour en revenir au benchmark, permets moi de te donner quelques réflexions (pas par odre d'importance, mais dans l'ordre où mon cerveau les produit !) :

1. Un benchmark reste une approximation, à ne jamais oublier !

2. Ce que tu proposes fait plus appel aux API Palm OS, ce n'est donc plus tout à fait le langage de développement qui est benchmarké mais plutôt Palm OS lui-même. Il y aura donc un énorme lissage entre les différents outils.

3. Un benchmark d'outil de développement doit donc en rester à des opérations mathématiques simples, en nombres entiers. Tu noteras mon paragraphe sur la routine nombres réels de Bench2... nombre d'outils s'appuient sur Palm OS pour les réels...

4. Un compilo pur et dur, rapide, sera vraiment utile pour ceux qui font beaucoup d'algorithmique et de calculs dans leur logiciel, et il y en a plus que tu le crois... ne serait-ce que nombre de jeux. Si tu prends mes deux jeux Cells et PalmPulus, ils ne tournent de façon fluide que parce que j'ai pris un des outils en haut du tableau pour chacun d'entre eux.


Reaction #14 

by miellaby 07/02/2005 @ 13:29

je me demande encore pourquoi je suis intervenu. On ne devrait jamais émettre une critique avant d'avoir formuler 7 compliments, pas vrai?

Donc j'ai vraiment apprécié ce benchmark, et surtout ta volonté de faire partager ta connaissance pour faire avancer le schmilblick.

Et puis, les sources dans les différents langages. C'est très éducatif.

J'ai tout de suite mis cette page et bien d'autre dans mes signets.

En plus, après avoir installer pp (influencé par le benchmark), j'ai explorer et tester le source de NekoPP.

Quel talent! Bravi brava bravo.

Mais bon, j'en reviens à l'interprétation des résultats du benchmark. Je veux pas te faire de la peine, mais si ça s'adresse à un développeur, c'est un développeur franchement atypique!

... parce que sur le Palm d'un invidu Lambda, y'a plus une seule appli qui ait besoin de calculer une factorielle. Quand tu dis que l'objectif est de tester "une routine pas spécialement optimisée"... j'aurai aimé que tu ajoutes :"[...] et qui ne correspond pas à un cas d'utilisation courant des langages aujourd'hui".

Si tu fais un benchmark sur la base d'un petit gestionnaire d'adresse (ajouter/supprimer/chercher une adresse), les langages scriptiques qui savent gérer en natif des collections de donnée se retrouvent en tête, avec un code super simpe en prime. Ben c'est tout. Fallait le dire non?

StartPrevious [ 1 2 3 4 5 6 ] NextEnd
Special !

Mobile version of
the aldweb Site

Neither French, nor English?

Try one of these flags then:
[de] [es] [it] [pt]
Weather / Spam / www

Lyon Weather

aldweb against spam

Family's web sites

Friends News
What do you think of the new design of the aldweb Site?
Great !
So ugly...
^ Top ^