This report aims to provide insight into the functioning of the experimental Quality statistic, which is meant to be an indicator of overall test quality. The statistic is computed as the "root mean square" average of the six primary test statistics, to wit quality of norms, sample-independent hardness, resolution, estimated g factor loading, reliability, and robustness, which are all scaled from 0 to 1.
This table show the Quality values that have been computed so far. Test names have been abbreviated to keep the table from getting too large.
| .30-.39 | .40-.49 | .50-.59 | .60-.69 | .70-.79 | .80-.89 |
|---|---|---|---|---|---|
| QAT #1 .31 |
GRIT (old) .42 QAT #3 .44 QMC #2 .44 |
QAT #2 .51 LIMIT .55 SIT .56 Bonsai .57 NIT .58 COLT .59 |
Odds .60 QMC #1 .60 TFG - Rev. 2010 - Num. S. .61 Evens .61 LTFG - German .62 Words .62 Association subtest of LTFG .63 Sargasso .63 TFG - Rev. 2010 - Num. .64 Analogies subtest of LTFG .64 GRIT .65 Isis .65 Association subtest of LTFG - NL .65 Letters .65 Analogies #1 .66 Numbers .66 Shock and Awe .66 Analogies subtest of LTFG - NL .67 CIT - Form 1 .67 LAW .67 QMC #3 .67 QMC #4 .67 Reason .67 LTFG - French .68 |
Marathon - Spatial .70 Laatste Test .71 TFG - Rev. 2004 - Spatial .71 TFG - Rev. 2004 .71 CIT - Form 4 .73 GAT .73 Reason - Rev. 2008 .73 TFG - Rev. 2010 .74 TFG - Rev. 2004 - Verbal 74 Cartoons of Shock .74 PIGS first degree .75 Final Test .75 RBMC .75 TTEAT .76 VIT .76 STH .77 Beheaded Man .78 CIT - Form 3 .78 LTFG - NL .78 Marathon - Num. and S. .79 Marathon - Num. .79 QMC #5 .79 |
COLT 2 .80 Nemesis .80 STFG .80 CIT - Form 2 .80 Daedalus .82 RBMC - Rev. 2008 .82 Giga Test .82 LTFG .82 |
A rough tendency that can be observed is that short and/or homogeneous (one-sided) tests have lower Quality scores while long and/or heterogeneous tests have higher ones.
An observation regarding the resolution of this statistic is that, often, several tests have the same score to a hundredth, which suggests that a third decimal is needed to discriminate between them. This is also logical, considering that the primary statistics are all expressed to two decimals, while a general rule in statistics is that an average (or anything involving division) should be expressed to a decimal more than the values from which it is computed. In future reports therefore, the Quality statistic will be give to three decimals.