Quality of norms reflects both the number of score pairs used in norming (N) and their weighted mean correlation (r) with the object test:
Quality of norms = √N × r2
When selecting other tests for norming the object test, the threshold correlation is normally set so as to maximize the quality of norms; that is, it is set where the expression √N × r2 has its maximum (for tests with little data, the threshold may be set lower though if there is too little data for norming otherwise). The result is that mainly tests with high correlations will be selected, but also a smaller number of tests with lower correlations. There has been some experimentation regarding whether the correlation should be squared or not, and it was decided to square, for with the unsquared correlation, a bit too many lower-quality tests and low-correlating tests would be included.
The scaling to a number from 0 to 1 as previously employed has been abandoned for being too arbitrary; the divider had to be adjusted frequently as more data became available.
Quality of norms reflects both the number of score pairs used in norming (N) and their mean correlation (r) with the object test; this statistic is in itself the best indicator of quality of norms:
Quality of norms (unscaled) = √N × r2
When selecting other tests for norming the object test, the threshold correlation is set so as to maximize the quality of norms; that is, it is set where the expression √N × r2 has its maximum. The result is that mainly tests with high correlations will be selected, and tests with low correlations will only be included when there are very many of them.
To allow combination of this statistic with other measures of test quality, a scaling from 0 to 1 is also provided as follows:
Quality of norms (scaled) = (Quality of norms (unscaled)) / divider
This method replaces the old one, the scaled values of which were becoming too high, and therefore less informative, as a result of the increasing amount of available norming data. The new scaled values are lower than the old ones. Whenever they become inflated in the future, the divider may be increased to solve this.
Quality of norms reflects both the number of score pairs used in norming and their correlations with the object test; the weighted sum of correlations of the used tests is in itself the best indicator of quality of norms. To allow combination of this statistic with other measures of test quality, a scaling from 0 to 1 is also provided as follows:
Quality of norms (scaled) = √(weighted sum of correlations) / 25
This method replaces the old one, the scaled values of which were becoming too high, and therefore less informative, as a result of the increasing amount of available norming data. The new scaled values are much lower than the old ones. Whenever they become inflated in the future, the divider may be increased to solve this.