Intraindividual variation in high-range I.Q. test scores

© November 2019 Paul Cooijmans

Introduction

An important decision in studying the spread of scores within the individual is how to set the "minimum number of tests taken" threshold for inclusion of candidates in the sample. This value has significant qualitative and quantitative consequences for the results, and these must be understood before reading the report. The following considerations are at play:

  1. It takes several tests for the spread of a candidate's scores to "settle", especially in case of the range. The intraindividual standard deviation is somewhat more stable than the range across "number of tests taken". This can be observed in the earlier report Intraindividual variation in high-range I.Q. test scores by number of tests taken. The (outlier-resistant) quartile deviation has been considered as an alternative for the standard deviation, but the latter was preferred because it turned out more stable across "number of tests taken"; in particular, the quartile deviation for 3 tests taken yields an unrealistically high value, so the threshold would need to be moved to 4, thus drastically reducing the number of available candidates.
  2. The "minimum number of tests taken" threshold has vast consequences for the number of candidates included in the sample, because the number of available candidates decreases sharply with "number of tests taken". This can be observed in the report Number of scores per candidate. Lowering the threshold from 3 to 2 would yield about 160 extra candidates (so about 320 extra scores). But since those two-time test takers have smaller spreads (especially smaller ranges) they would significantly affect the resulting statistics.

As a reasonable compromise, it has been decided to set the minimum number of tests taken at 3 for this report and use the standard deviation and range as measures of intraindividual spread.

Goals

The goals of this report are (1) to give insight in the intraindividual spread of scores of repetitive test takers, and (2) to look for possible correlations of that spread with personality variables. Do certain traits go with a larger spread of scores? There is an intuitive anecdotal notion that very large spreads are related to being less conscientious, to being mentally unstable, to fraud, and to purposely underperforming, but this is the first time it has been formally studied.

Intraindividual spread of protonorm scores for candidates having taken at least 3 tests

For clarity, "at least 3 tests" refers to the number of tests taken in the pertinent category, for instance "Heterogeneous".

Test type:HeterogeneousHomogeneousAll tests combined
Number of candidates180312472
Total number of scores111917233567
Median of the candidates' median scores403.5410.8403.0
Median of the candidates' intraindividual standard deviations45.250.147.0
Median of the candidates' intraindividual ranges107.0120.5120.0

All score-related values in this report are in protonorms (score level) or protonorm points (interval, difference); keep in mind that
1 protonorm point ≈ 0.15 I.Q. points;
1 I.Q. point ≈ 6.67 protonorm points.

The main observation that can be made in the above table is that heterogeneous tests have a smaller intraindividual spread than homogeneous tests, both in terms of standard deviation and range. This observation also holds true for the quartile deviation (not included here) and also holds true when any other measure of centrality is computed across the intraindividual standard deviations, ranges, or quartile deviations (like the mean or the root-mean-square average). This observation lends support to the policy of only using and recognizing heterogeneous tests as indicators of general intelligence and I.Q. It implies that scores on heterogeneous tests, having a smaller spread, necessarily deviate less from the candidate's "true" score. It implies that heterogeneous tests, on the whole, have a smaller error of measurement with regard to g.

Since this is the first time this difference between heterogeneous and homogeneous tests has been made visible, these conclusions should not be regarded as final. It will need to be examined again on later occasions, with possible improvements to the method and more or other data. There are many ways to arrive at a table as the above one, using different measures of spread, different ways of averaging the intraindividual results, different values of "minimum number of tests taken", and different ways of candidate selection for comparing heterogeneous with homogeneous tests (like enforcing an equal number of candidates, an equal number of tests, or even that they are the very same candidates, but all of that has its problems and disadvantages).

Correlates of intraindividual spread

Correlations of intraindividual standard deviation with personalia (heterogeneous tests)

Personaliumrn
Sex0.11179
Birth year0.09179
Observed associative horizon-0.0323
Educational level-0.05160
Disorders (own)0.18159
P.S.I.A. Antisocial0.1047
P.S.I.A. Aspergoid0.0547
P.S.I.A. Cold0.1247
P.S.I.A. Cruel-0.1147
P.S.I.A. Extreme0.0847
P.S.I.A. Introverted0.0147
P.S.I.A. Just0.2547
P.S.I.A. Neurotic0.0047
P.S.I.A. Orderly-0.1247
P.S.I.A. Rare-0.0447
P.S.I.A. Rational0.4747
P.S.I.A. True0.0747
P.S.I.A. Deviant0.0960
P.S.I.A. Ethical0.1060
P.S.I.A. System0.0939
Observed behaviour-0.0745
Gifted Adult's Inventory of Aspergerisms0.1640
Cooijmans Inventory of Neo-Marxist Attitudes-0.0716

P.S.I.A. Rational has a significant positive correlation with the intraindividual standard deviation of scores on heterogeneous tests, while P.S.I.A. Just is almost significant.

Correlations of intraindividual range with personalia (heterogeneous tests)

Personaliumrn
Sex0.11179
Birth year0.10179
Observed associative horizon-0.0123
Educational level-0.10160
Disorders (own)0.13159
P.S.I.A. Antisocial0.1247
P.S.I.A. Aspergoid0.0147
P.S.I.A. Cold0.2247
P.S.I.A. Cruel-0.1047
P.S.I.A. Extreme0.1147
P.S.I.A. Introverted-0.0047
P.S.I.A. Just0.3147
P.S.I.A. Neurotic-0.1147
P.S.I.A. Orderly0.0347
P.S.I.A. Rare-0.1047
P.S.I.A. Rational0.2847
P.S.I.A. True0.0347
P.S.I.A. Deviant0.0360
P.S.I.A. Ethical0.0560
P.S.I.A. System0.0339
Observed behaviour-0.0745
Gifted Adult's Inventory of Aspergerisms0.1840
Cooijmans Inventory of Neo-Marxist Attitudes-0.2116

When the intraindividual range is considered, P.S.I.A. Just and Rational are again the highest correlates. This somewhat puzzling, and one should realize that this table contains 23 correlations, so that, by mere chance, one could expect one or two significant correlations among them. The proper conclusion here may be that there is no good evidence that the spread of scores on heterogeneous tests is related to any particular personality traits. The spread may be a random phenomenon.

Correlations of intraindividual standard deviation with personalia (homogeneous tests)

Personaliumrn
Sex0.04310
Birth year0.04301
Observed associative horizon-0.0434
Educational level0.03213
Disorders (own)0.08210
P.S.I.A. Antisocial0.0973
P.S.I.A. Aspergoid0.3273
P.S.I.A. Cold0.1873
P.S.I.A. Cruel-0.1473
P.S.I.A. Extreme0.0473
P.S.I.A. Introverted0.2473
P.S.I.A. Just0.1473
P.S.I.A. Neurotic0.3973
P.S.I.A. Orderly-0.1173
P.S.I.A. Rare0.2273
P.S.I.A. Rational0.2473
P.S.I.A. True0.1573
P.S.I.A. Deviant0.24103
P.S.I.A. Ethical0.19103
P.S.I.A. System0.2463
Observed behaviour-0.2160
Gifted Adult's Inventory of Aspergerisms0.1754
Cooijmans Inventory of Neo-Marxist Attitudes-0.3015

Regarding homogeneous tests, the pattern of correlations is different, P.S.I.A. Neurotic and P.S.I.A. Apergoid being the top correlates of intraindividual standard deviation.

Correlations of intraindividual range with personalia (homogeneous tests)

Personaliumrn
Sex0.05310
Birth year0.08301
Observed associative horizon-0.0634
Educational level-0.08213
Disorders (own)0.06210
P.S.I.A. Antisocial0.0673
P.S.I.A. Aspergoid0.2873
P.S.I.A. Cold0.2573
P.S.I.A. Cruel-0.1573
P.S.I.A. Extreme0.0373
P.S.I.A. Introverted0.1973
P.S.I.A. Just0.2473
P.S.I.A. Neurotic0.2973
P.S.I.A. Orderly-0.0773
P.S.I.A. Rare0.1673
P.S.I.A. Rational0.2473
P.S.I.A. True0.1773
P.S.I.A. Deviant0.18103
P.S.I.A. Ethical0.18103
P.S.I.A. System0.1863
Observed behaviour-0.2260
Gifted Adult's Inventory of Aspergerisms0.1554
Cooijmans Inventory of Neo-Marxist Attitudes-0.4515

The intraindividual range of scores on homogeneous tests shows a similar pattern as was apparent with the standard deviation, P.S.I.A. Neurotic being the highest in both cases. The number of pairs (73) is greater than was the case with the top correlates of spread on heterogeneous test scores. On the whole, therefore, there is some evidence that persons higher on neuroticism have their scores further apart.

Correlations of intraindividual standard deviation with personalia (all tests)

Personaliumrn
Sex0.02469
Birth year0.01457
Observed associative horizon-0.0641
Educational level0.03344
Disorders (own)0.11345
P.S.I.A. Antisocial0.1098
P.S.I.A. Aspergoid0.1998
P.S.I.A. Cold0.1198
P.S.I.A. Cruel-0.0698
P.S.I.A. Extreme0.0298
P.S.I.A. Introverted0.1498
P.S.I.A. Just0.2098
P.S.I.A. Neurotic0.1598
P.S.I.A. Orderly0.0598
P.S.I.A. Rare0.1698
P.S.I.A. Rational0.3298
P.S.I.A. True0.0898
P.S.I.A. Deviant0.19137
P.S.I.A. Ethical0.14137
P.S.I.A. System0.1983
Observed behaviour-0.1785
Gifted Adult's Inventory of Aspergerisms0.1870
Cooijmans Inventory of Neo-Marxist Attitudes-0.3126

Correlations of intraindividual range with personalia (all tests)

Personaliumrn
Sex0.04469
Birth year0.05457
Observed associative horizon-0.1541
Educational level0.00344
Disorders (own)0.11345
P.S.I.A. Antisocial0.1498
P.S.I.A. Aspergoid0.1698
P.S.I.A. Cold0.1698
P.S.I.A. Cruel-0.0898
P.S.I.A. Extreme0.0198
P.S.I.A. Introverted0.0998
P.S.I.A. Just0.2598
P.S.I.A. Neurotic0.1398
P.S.I.A. Orderly0.0198
P.S.I.A. Rare0.0898
P.S.I.A. Rational0.2598
P.S.I.A. True0.0698
P.S.I.A. Deviant0.13137
P.S.I.A. Ethical0.12137
P.S.I.A. System0.1383
Observed behaviour-0.1785
Gifted Adult's Inventory of Aspergerisms0.1970
Cooijmans Inventory of Neo-Marxist Attitudes-0.4126

With homogeneous and heterogeneous test combined, the variables that differed between them are naturally evened out a bit. What remains is an almost significant negative correlation with Neo-Marxist Attitudes, which, if real, would mean that persons high in neo-Marxist attitudes have their scores closer together.

Conclusions and observations

  1. There is fairly convincing evidence that a candidate's scores on heterogeneous tests have a smaller spread than those on homogeneous tests;
  2. The pattern of correlations of intraindividual spread with personalia is different for heterogeneous than for homogeneous tests;
  3. The intraindividual spread on homogeneous tests may be positively correlated with neuroticism;
  4. The evidence for any significant correlations of the spread on heterogeneous tests with personalia is not convincing; perhaps, heterogeneous tests are by and large neutral across different personality profiles.

Histograms of standard deviations and ranges for the test categories

For information, here are the actual standard deviations (rounded to nearest whole number) and ranges per candidate.

Histogram of the intraindividual candidates' standard deviations (rounded) (heterogeneous tests)

1 *
2 *
6 **
9 *
10 **
12 **
13 *
14 ***
15 *
16 **
17 ***
18 **
19 ***
20 **
21 *
22 ***
24 *
25 ****
26 ***
27 ***
28 ***
29 *
30 **
31 ***
32 *
33 ****
34 **
35 ******
36 ***
37 **
38 **
39 **
40 ***
41 ***
42 ***
43 *
44 *****
45 ****
46 ********
47 **
48 ***
50 ******
51 ***
52 ***
53 **
55 **
56 ***
57 ***
58 **
59 ***
60 ****
61 ****
62 ***
63 *
64 **
66 *
67 *
68 *
69 *
70 *
71 *
76 **
77 *
79 *
80 *
81 *
82 *
83 *
84 **
88 *
91 *
94 **
95 ***
96 **
104 *
106 *
107 *
108 *
109 *
115 **
120 *
134 *
137 *

Histogram of the intraindividual candidates' ranges (heterogeneous tests)

1 *
3 *
11 *
12 *
18 *
23 **
25 *
26 *
27 *
28 *
29 *
30 *
31 **
32 *
34 *
35 **
37 *
40 *
49 *
50 ***
51 *
53 *
54 ****
55 **
56 *
58 *
59 **
60 **
61 *
63 ***
65 *
66 **
67 ***
70 *
71 *
75 *
76 ****
77 *
78 **
80 **
81 ****
83 **
85 **
86 *
87 *
90 **
91 *
92 *
94 *
95 ***
96 *
99 **
100 **
101 *
103 *
104 *
105 *
106 *
107 **
108 *
109 *
110 **
111 *
112 ****
114 *
115 **
118 *
119 *
120 **
121 *
122 *
124 *
126 **
127 *
128 **
131 *
132 *
133 **
134 *
135 ***
139 *
143 *
144 *
150 *
153 *
155 *
157 **
160 **
163 *
169 *
172 *
176 *
177 *
179 *
180 *
181 *
189 **
192 *
194 *
195 *
205 **
210 *
212 *
213 *
215 *
216 *
217 *
218 *
220 *
224 *
226 *
228 **
233 *
234 **
235 *
237 *
238 *
241 *
243 *
249 ***
251 *
253 *
264 *
273 *
275 *
276 *
280 *
313 *
317 *

Histogram of the intraindividual candidates' standard deviations (rounded) (homogeneous tests)

3 **
4 **
9 **
11 ***
12 *
13 *
14 ****
15 *
16 *****
17 ****
18 ***
19 **
20 ***
21 ****
22 **
23 **
24 **
25 ****
26 *
27 ****
28 *
29 *****
30 *
31 ****
32 *******
33 ****
34 ***
35 ***
36 ******
37 *****
38 *****
39 ******
40 ***
41 ******
42 **
43 *******
44 ******
45 ***
46 ******
47 ****
48 ******
49 *****
50 ********
51 ****
52 *******
54 **********
55 ****
56 *****
57 **
58 ***
59 *****
60 ***
61 ****
62 ******
63 ****
64 **
65 *******
66 ****
67 **
68 **
69 *****
70 ****
71 **
72 *
73 *****
74 ***
75 **
76 *
77 **
78 ***
79 ****
80 *
81 *
82 ***
83 ***
84 ***
85 **
86 *
88 *
89 *
90 *
92 **
93 *
94 *
96 **
97 **
99 *****
101 *
102 *
103 **
104 **
105 **
111 *
112 *
114 *
119 *
120 *
125 *
135 *
136 *
144 *
333 *

Histogram of the intraindividual candidates' ranges (homogeneous tests)

6 **
7 **
17 *
18 *
21 *
22 **
25 *
26 **
27 **
28 ***
29 **
31 *
33 *
35 *
36 *
38 *
39 *
40 ***
41 *
42 *
45 *
46 ****
47 ****
49 *
50 *
51 *
52 *
53 **
56 *
60 *****
62 ***
65 **
66 ***
67 **
70 **
72 *
73 ***
74 *****
76 ***
77 **
78 *
79 *
80 ***
81 *
82 *
83 *
84 *
85 *
86 ***
87 ******
88 *
90 *
91 **
92 *
93 ****
94 **
95 *
96 *
97 *
98 *
99 ***
100 *****
101 *
104 *
105 **
106 **
107 ***
109 *****
111 **
112 **
113 ***
114 ***
115 **
117 ****
119 **
120 *****
121 *
122 **
125 ***
126 ****
127 ****
128 *
129 ***
130 *
133 **
134 **
135 *
136 **
137 **
139 **
140 ********
141 *
144 *
145 *
146 *
147 ******
149 **
152 *
153 ****
154 ******
155 **
156 *
158 *
159 *
160 *
163 *
165 *
166 *
167 *****
168 *
171 **
172 *
173 *
174 *
176 **
179 **
180 *******
183 *
184 *
185 **
188 **
191 ***
192 **
194 **
196 *
198 *
199 *
200 ***
201 **
202 **
206 *
208 *
211 *
212 *
216 **
220 *
222 *
223 *
225 *
226 *
229 *
230 **
233 *
234 *
236 *
240 **
256 *
259 *
263 **
267 *
270 *
273 **
274 *
276 *
278 *
281 *
286 *
288 *
310 *
311 *
320 *
330 *
332 *
352 *
445 *
753 *

Histogram of the intraindividual candidates' standard deviations (rounded) (all tests)

1 *
2 *
3 **
4 ***
6 ***
8 *
9 *
11 ****
12 ******
13 *
14 ****
15 *****
16 ****
17 *******
18 ****
19 ****
20 ****
21 *****
22 **
23 *****
24 ******
25 ***********
26 ****
27 ***********
28 **********
29 *************
30 ***
31 ******
32 *********
33 ****
34 *********
35 *******
36 *****
37 ****
38 *****
39 **
40 ********
41 **********
42 *******
43 *******
44 *********
45 **********
46 *****
47 ********
48 ******
49 *********
50 *********
51 ************
52 **********
53 *****
54 ******
55 ******
56 ******
57 ******
58 ****
59 ****
60 *******
61 *********
62 *********
63 ***
64 *****
65 *******
66 **
67 *****
68 ****
69 *****
70 ***
71 ***
72 *****
73 ***
74 ****
75 **
76 **
77 ******
78 ****
79 **
80 ***
81 *
82 *
83 ******
84 **
85 **
86 *
87 ***
88 **
90 **
91 **
92 ****
93 **
94 *
96 *
97 ****
98 *
99 *
100 **
103 **
105 *
108 *
109 *
110 **
113 *
117 *
118 *
119 *
122 *
139 *
141 *
168 *
174 *
230 *
264 *

Histogram of the intraindividual candidates' ranges (all tests)

1 *
3 *
6 **
7 ***
11 *
12 **
20 **
21 **
22 **
25 *
26 ***
27 *
28 ****
29 **
30 *
31 *
32 **
33 **
34 *
35 **
36 **
38 **
39 *
40 ***
41 **
42 ***
44 **
45 *
46 ******
47 ***
48 *
49 *
50 ***
51 *
52 **
53 ***
54 ***
56 ***
57 **
58 *
59 **
60 ****
62 *
63 **
65 ***
66 ******
67 *****
68 *
70 *
72 ***
73 ***
74 **
75 ****
76 **
78 **
80 ******
81 **
82 **
83 ***
85 ****
86 ********
87 *********
90 ****
91 ***
92 **
93 ******
94 ****
95 ***
97 **
98 ***
99 **
100 ******
101 **
103 **
104 **
105 ****
106 ***
107 ***
108 ***
109 **
111 **
112 *****
113 ****
114 ***
115 *
116 *
117 **
119 ***
120 *********
122 ***
123 **
124 *
125 **
126 **
127 *****
128 *
129 ****
130 *
132 *
133 *****
134 *
135 **
136 ***
137 **
139 **
140 ********
141 **
142 *
143 *
144 **
145 *
146 ***
147 ******
149 *
150 *
152 **
153 ****
154 ******
155 **
156 *
158 **
159 *
160 *
161 *
163 *
164 *
165 *
166 *
167 *****
168 *
169 *
170 *
173 **
176 **
177 **
179 ****
180 ********
182 **
183 ***
184 *
185 ***
186 **
187 *
188 ***
189 **
191 ****
192 *
194 *
195 *
197 *
198 **
199 **
200 ***
201 *
202 **
204 ***
206 **
208 *
210 *
211 *
212 *
215 *
216 **
217 *
218 *
219 *
220 ***
223 *
224 **
225 *
226 *
228 *
229 *
230 **
233 *
234 **
236 *
238 **
240 *****
245 *
249 *
250 *
251 *
252 *
254 **
256 *
257 *
259 **
263 **
266 *
268 *
270 *
273 ***
276 **
278 *
280 *
286 *
290 *
293 *
294 *
306 *
307 **
310 *
317 **
320 *
321 *
325 *
330 *
332 *
342 *
344 *
352 *
415 *
445 *
753 *