[cap-talk] evaluating tools

[David Wagner]

Zooko writes:
>I reconcile it by believing that the dominant platform for web  
>services in the enterprise world is a technically inferior platform.
>
>One shouldn't think that market dominance implies technical  
>superiority.

Sure.  So one could potentially describe Java's performance
as "inferior but acceptable".  That lends a different impression
than "dreadful", doesn't it?  My point is that it's better to use
hard numbers, not emotion-laden adjectives, to describe what is
essentially a quantitative phenomenom.  The choice of adjectives
is too easily influenced by personal preferences.

>It would be really good if we could quantify claims such as "Tool X  
>is technically superior for task Y.".  Presumably we could do that  
>using the decades-old practices of controlled trials, such as are  
>used in psychology, human factors engineering, and so on.   
>Unfortunately such an experiment would be expensive, and few people  
>have attempted to run one.  Here is the best example that I have found:
>
>"An empirical comparison of C, C++, Java, Perl, Python, Rexx, and Tcl  
>for a search/string-processing program" (2000) by Lutz Prechelt
>http://citeseer.ist.psu.edu/311372.html
>
>It is unfortunate that designers of programming languages often  
>consider themselves to be working in the fields of mathematics or  
>computer science, and they use the practices and tools of those arts,  
>when we would be better served if they thought of themselves as  
>working in psychology and used the standards of measurement required  
>for publishing results in *that* field.

Interesting point!