Intraindividual variation in high-range I.Q. test scores

Introduction

An important decision in studying the spread of scores within the individual is how to set the "minimum number of tests taken" threshold for inclusion of candidates in the sample. This value has significant qualitative and quantitative consequences for the results, and these must be understood before reading the report. The following considerations are at play:

It takes several tests for the spread of a candidate's scores to "settle", especially in case of the range. The intraindividual standard deviation is somewhat more stable than the range across "number of tests taken". This can be observed in the earlier report Intraindividual variation in high-range I.Q. test scores by number of tests taken. The (outlier-resistant) quartile deviation has been considered as an alternative for the standard deviation, but the latter was preferred because it turned out more stable across "number of tests taken"; in particular, the quartile deviation for 3 tests taken yields an unrealistically high value, so the threshold would need to be moved to 4, thus drastically reducing the number of available candidates.
The "minimum number of tests taken" threshold has vast consequences for the number of candidates included in the sample, because the number of available candidates decreases sharply with "number of tests taken". This can be observed in the report Number of scores per candidate. Lowering the threshold from 3 to 2 would yield about 160 extra candidates (so about 320 extra scores). But since those two-time test takers have smaller spreads (especially smaller ranges) they would significantly affect the resulting statistics.

As a reasonable compromise, it has been decided to set the minimum number of tests taken at 3 for this report and use the standard deviation and range as measures of intraindividual spread.

Goals

The goals of this report are (1) to give insight in the intraindividual spread of scores of repetitive test takers, and (2) to look for possible correlations of that spread with personality variables. Do certain traits go with a larger spread of scores? There is an intuitive anecdotal notion that very large spreads are related to being less conscientious, to being mentally unstable, to fraud, and to purposely underperforming, but this is the first time it has been formally studied.

Intraindividual spread of protonorm scores for candidates having taken at least 3 tests

For clarity, "at least 3 tests" refers to the number of tests taken in the pertinent category, for instance "Heterogeneous".

Test type:	Heterogeneous	Homogeneous	All tests combined
Number of candidates	180	312	472
Total number of scores	1119	1723	3567
Median of the candidates' median scores	403.5	410.8	403.0
Median of the candidates' intraindividual standard deviations	45.2	50.1	47.0
Median of the candidates' intraindividual ranges	107.0	120.5	120.0

All score-related values in this report are in protonorms (score level) or protonorm points (interval, difference); keep in mind that
1 protonorm point ≈ 0.15 I.Q. points;
1 I.Q. point ≈ 6.67 protonorm points.

The main observation that can be made in the above table is that heterogeneous tests have a smaller intraindividual spread than homogeneous tests, both in terms of standard deviation and range. This observation also holds true for the quartile deviation (not included here) and also holds true when any other measure of centrality is computed across the intraindividual standard deviations, ranges, or quartile deviations (like the mean or the root-mean-square average). This observation lends support to the policy of only using and recognizing heterogeneous tests as indicators of general intelligence and I.Q. It implies that scores on heterogeneous tests, having a smaller spread, necessarily deviate less from the candidate's "true" score. It implies that heterogeneous tests, on the whole, have a smaller error of measurement with regard to g.

Since this is the first time this difference between heterogeneous and homogeneous tests has been made visible, these conclusions should not be regarded as final. It will need to be examined again on later occasions, with possible improvements to the method and more or other data. There are many ways to arrive at a table as the above one, using different measures of spread, different ways of averaging the intraindividual results, different values of "minimum number of tests taken", and different ways of candidate selection for comparing heterogeneous with homogeneous tests (like enforcing an equal number of candidates, an equal number of tests, or even that they are the very same candidates, but all of that has its problems and disadvantages).

Correlates of intraindividual spread

Correlations of intraindividual standard deviation with personalia (heterogeneous tests)

Personalium	r	n
Sex	0.11	179
Birth year	0.09	179
Observed associative horizon	-0.03	23
Educational level	-0.05	160
Disorders (own)	0.18	159
P.S.I.A. Antisocial	0.10	47
P.S.I.A. Aspergoid	0.05	47
P.S.I.A. Cold	0.12	47
P.S.I.A. Cruel	-0.11	47
P.S.I.A. Extreme	0.08	47
P.S.I.A. Introverted	0.01	47
P.S.I.A. Just	0.25	47
P.S.I.A. Neurotic	0.00	47
P.S.I.A. Orderly	-0.12	47
P.S.I.A. Rare	-0.04	47
P.S.I.A. Rational	0.47	47
P.S.I.A. True	0.07	47
P.S.I.A. Deviant	0.09	60
P.S.I.A. Ethical	0.10	60
P.S.I.A. System	0.09	39
Observed behaviour	-0.07	45
Gifted Adult's Inventory of Aspergerisms	0.16	40
Cooijmans Inventory of Neo-Marxist Attitudes	-0.07	16

P.S.I.A. Rational has a significant positive correlation with the intraindividual standard deviation of scores on heterogeneous tests, while P.S.I.A. Just is almost significant.

Correlations of intraindividual range with personalia (heterogeneous tests)

Personalium	r	n
Sex	0.11	179
Birth year	0.10	179
Observed associative horizon	-0.01	23
Educational level	-0.10	160
Disorders (own)	0.13	159
P.S.I.A. Antisocial	0.12	47
P.S.I.A. Aspergoid	0.01	47
P.S.I.A. Cold	0.22	47
P.S.I.A. Cruel	-0.10	47
P.S.I.A. Extreme	0.11	47
P.S.I.A. Introverted	-0.00	47
P.S.I.A. Just	0.31	47
P.S.I.A. Neurotic	-0.11	47
P.S.I.A. Orderly	0.03	47
P.S.I.A. Rare	-0.10	47
P.S.I.A. Rational	0.28	47
P.S.I.A. True	0.03	47
P.S.I.A. Deviant	0.03	60
P.S.I.A. Ethical	0.05	60
P.S.I.A. System	0.03	39
Observed behaviour	-0.07	45
Gifted Adult's Inventory of Aspergerisms	0.18	40
Cooijmans Inventory of Neo-Marxist Attitudes	-0.21	16

When the intraindividual range is considered, P.S.I.A. Just and Rational are again the highest correlates. This somewhat puzzling, and one should realize that this table contains 23 correlations, so that, by mere chance, one could expect one or two significant correlations among them. The proper conclusion here may be that there is no good evidence that the spread of scores on heterogeneous tests is related to any particular personality traits. The spread may be a random phenomenon.

Correlations of intraindividual standard deviation with personalia (homogeneous tests)

Personalium	r	n
Sex	0.04	310
Birth year	0.04	301
Observed associative horizon	-0.04	34
Educational level	0.03	213
Disorders (own)	0.08	210
P.S.I.A. Antisocial	0.09	73
P.S.I.A. Aspergoid	0.32	73
P.S.I.A. Cold	0.18	73
P.S.I.A. Cruel	-0.14	73
P.S.I.A. Extreme	0.04	73
P.S.I.A. Introverted	0.24	73
P.S.I.A. Just	0.14	73
P.S.I.A. Neurotic	0.39	73
P.S.I.A. Orderly	-0.11	73
P.S.I.A. Rare	0.22	73
P.S.I.A. Rational	0.24	73
P.S.I.A. True	0.15	73
P.S.I.A. Deviant	0.24	103
P.S.I.A. Ethical	0.19	103
P.S.I.A. System	0.24	63
Observed behaviour	-0.21	60
Gifted Adult's Inventory of Aspergerisms	0.17	54
Cooijmans Inventory of Neo-Marxist Attitudes	-0.30	15

Regarding homogeneous tests, the pattern of correlations is different, P.S.I.A. Neurotic and P.S.I.A. Aspergoid being the top correlates of intraindividual standard deviation.

Correlations of intraindividual range with personalia (homogeneous tests)

Personalium	r	n
Sex	0.05	310
Birth year	0.08	301
Observed associative horizon	-0.06	34
Educational level	-0.08	213
Disorders (own)	0.06	210
P.S.I.A. Antisocial	0.06	73
P.S.I.A. Aspergoid	0.28	73
P.S.I.A. Cold	0.25	73
P.S.I.A. Cruel	-0.15	73
P.S.I.A. Extreme	0.03	73
P.S.I.A. Introverted	0.19	73
P.S.I.A. Just	0.24	73
P.S.I.A. Neurotic	0.29	73
P.S.I.A. Orderly	-0.07	73
P.S.I.A. Rare	0.16	73
P.S.I.A. Rational	0.24	73
P.S.I.A. True	0.17	73
P.S.I.A. Deviant	0.18	103
P.S.I.A. Ethical	0.18	103
P.S.I.A. System	0.18	63
Observed behaviour	-0.22	60
Gifted Adult's Inventory of Aspergerisms	0.15	54
Cooijmans Inventory of Neo-Marxist Attitudes	-0.45	15

The intraindividual range of scores on homogeneous tests shows a similar pattern as was apparent with the standard deviation, P.S.I.A. Neurotic being the highest in both cases. The number of pairs (73) is greater than was the case with the top correlates of spread on heterogeneous test scores. On the whole, therefore, there is some evidence that persons higher on neuroticism have their scores further apart.

Correlations of intraindividual standard deviation with personalia (all tests)

Personalium	r	n
Sex	0.02	469
Birth year	0.01	457
Observed associative horizon	-0.06	41
Educational level	0.03	344
Disorders (own)	0.11	345
P.S.I.A. Antisocial	0.10	98
P.S.I.A. Aspergoid	0.19	98
P.S.I.A. Cold	0.11	98
P.S.I.A. Cruel	-0.06	98
P.S.I.A. Extreme	0.02	98
P.S.I.A. Introverted	0.14	98
P.S.I.A. Just	0.20	98
P.S.I.A. Neurotic	0.15	98
P.S.I.A. Orderly	0.05	98
P.S.I.A. Rare	0.16	98
P.S.I.A. Rational	0.32	98
P.S.I.A. True	0.08	98
P.S.I.A. Deviant	0.19	137
P.S.I.A. Ethical	0.14	137
P.S.I.A. System	0.19	83
Observed behaviour	-0.17	85
Gifted Adult's Inventory of Aspergerisms	0.18	70
Cooijmans Inventory of Neo-Marxist Attitudes	-0.31	26

Correlations of intraindividual range with personalia (all tests)

Personalium	r	n
Sex	0.04	469
Birth year	0.05	457
Observed associative horizon	-0.15	41
Educational level	0.00	344
Disorders (own)	0.11	345
P.S.I.A. Antisocial	0.14	98
P.S.I.A. Aspergoid	0.16	98
P.S.I.A. Cold	0.16	98
P.S.I.A. Cruel	-0.08	98
P.S.I.A. Extreme	0.01	98
P.S.I.A. Introverted	0.09	98
P.S.I.A. Just	0.25	98
P.S.I.A. Neurotic	0.13	98
P.S.I.A. Orderly	0.01	98
P.S.I.A. Rare	0.08	98
P.S.I.A. Rational	0.25	98
P.S.I.A. True	0.06	98
P.S.I.A. Deviant	0.13	137
P.S.I.A. Ethical	0.12	137
P.S.I.A. System	0.13	83
Observed behaviour	-0.17	85
Gifted Adult's Inventory of Aspergerisms	0.19	70
Cooijmans Inventory of Neo-Marxist Attitudes	-0.41	26

With homogeneous and heterogeneous tests combined, the variables that differed between them are naturally evened out a bit. What remains is an almost significant negative correlation with Neo-Marxist Attitudes, which, if real, would mean that persons high in neo-Marxist attitudes have their scores closer together.

Conclusions and observations

There is fairly convincing evidence that a candidate's scores on heterogeneous tests have a smaller spread than those on homogeneous tests;
The pattern of correlations of intraindividual spread with personalia is different for heterogeneous than for homogeneous tests;
The intraindividual spread on homogeneous tests may be positively correlated with neuroticism;
The evidence for any significant correlations of the spread on heterogeneous tests with personalia is not convincing; perhaps, heterogeneous tests are by and large neutral across different personality profiles.

Histograms of standard deviations and ranges for the test categories

For information, here are the actual standard deviations (rounded to nearest whole number) and ranges per candidate.

Histogram of the intraindividual candidates' standard deviations (rounded) (heterogeneous tests)

1	*
2	*
6	**
9	*
10	**
12	**
13	*
14	***
15	*
16	**
17	***
18	**
19	***
20	**
21	*
22	***
24	*
25	****
26	***
27	***
28	***
29	*
30	**
31	***
32	*
33	****
34	**
35	******
36	***
37	**
38	**
39	**
40	***
41	***
42	***
43	*
44	*****
45	****
46	********
47	**
48	***
50	******
51	***
52	***
53	**
55	**
56	***
57	***
58	**
59	***
60	****
61	****
62	***
63	*
64	**
66	*
67	*
68	*
69	*
70	*
71	*
76	**
77	*
79	*
80	*
81	*
82	*
83	*
84	**
88	*
91	*
94	**
95	***
96	**
104	*
106	*
107	*
108	*
109	*
115	**
120	*
134	*
137	*

Histogram of the intraindividual candidates' ranges (heterogeneous tests)

1	*
3	*
11	*
12	*
18	*
23	**
25	*
26	*
27	*
28	*
29	*
30	*
31	**
32	*
34	*
35	**
37	*
40	*
49	*
50	***
51	*
53	*
54	****
55	**
56	*
58	*
59	**
60	**
61	*
63	***
65	*
66	**
67	***
70	*
71	*
75	*
76	****
77	*
78	**
80	**
81	****
83	**
85	**
86	*
87	*
90	**
91	*
92	*
94	*
95	***
96	*
99	**
100	**
101	*
103	*
104	*
105	*
106	*
107	**
108	*
109	*
110	**
111	*
112	****
114	*
115	**
118	*
119	*
120	**
121	*
122	*
124	*
126	**
127	*
128	**
131	*
132	*
133	**
134	*
135	***
139	*
143	*
144	*
150	*
153	*
155	*
157	**
160	**
163	*
169	*
172	*
176	*
177	*
179	*
180	*
181	*
189	**
192	*
194	*
195	*
205	**
210	*
212	*
213	*
215	*
216	*
217	*
218	*
220	*
224	*
226	*
228	**
233	*
234	**
235	*
237	*
238	*
241	*
243	*
249	***
251	*
253	*
264	*
273	*
275	*
276	*
280	*
313	*
317	*

Histogram of the intraindividual candidates' standard deviations (rounded) (homogeneous tests)

3	**
4	**
9	**
11	***
12	*
13	*
14	****
15	*
16	*****
17	****
18	***
19	**
20	***
21	****
22	**
23	**
24	**
25	****
26	*
27	****
28	*
29	*****
30	*
31	****
32	*******
33	****
34	***
35	***
36	******
37	*****
38	*****
39	******
40	***
41	******
42	**
43	*******
44	******
45	***
46	******
47	****
48	******
49	*****
50	********
51	****
52	*******
54	**********
55	****
56	*****
57	**
58	***
59	*****
60	***
61	****
62	******
63	****
64	**
65	*******
66	****
67	**
68	**
69	*****
70	****
71	**
72	*
73	*****
74	***
75	**
76	*
77	**
78	***
79	****
80	*
81	*
82	***
83	***
84	***
85	**
86	*
88	*
89	*
90	*
92	**
93	*
94	*
96	**
97	**
99	*****
101	*
102	*
103	**
104	**
105	**
111	*
112	*
114	*
119	*
120	*
125	*
135	*
136	*
144	*
333	*

Histogram of the intraindividual candidates' ranges (homogeneous tests)

6	**
7	**
17	*
18	*
21	*
22	**
25	*
26	**
27	**
28	***
29	**
31	*
33	*
35	*
36	*
38	*
39	*
40	***
41	*
42	*
45	*
46	****
47	****
49	*
50	*
51	*
52	*
53	**
56	*
60	*****
62	***
65	**
66	***
67	**
70	**
72	*
73	***
74	*****
76	***
77	**
78	*
79	*
80	***
81	*
82	*
83	*
84	*
85	*
86	***
87	******
88	*
90	*
91	**
92	*
93	****
94	**
95	*
96	*
97	*
98	*
99	***
100	*****
101	*
104	*
105	**
106	**
107	***
109	*****
111	**
112	**
113	***
114	***
115	**
117	****
119	**
120	*****
121	*
122	**
125	***
126	****
127	****
128	*
129	***
130	*
133	**
134	**
135	*
136	**
137	**
139	**
140	********
141	*
144	*
145	*
146	*
147	******
149	**
152	*
153	****
154	******
155	**
156	*
158	*
159	*
160	*
163	*
165	*
166	*
167	*****
168	*
171	**
172	*
173	*
174	*
176	**
179	**
180	*******
183	*
184	*
185	**
188	**
191	***
192	**
194	**
196	*
198	*
199	*
200	***
201	**
202	**
206	*
208	*
211	*
212	*
216	**
220	*
222	*
223	*
225	*
226	*
229	*
230	**
233	*
234	*
236	*
240	**
256	*
259	*
263	**
267	*
270	*
273	**
274	*
276	*
278	*
281	*
286	*
288	*
310	*
311	*
320	*
330	*
332	*
352	*
445	*
753	*

Histogram of the intraindividual candidates' standard deviations (rounded) (all tests)

1	*
2	*
3	**
4	***
6	***
8	*
9	*
11	****
12	******
13	*
14	****
15	*****
16	****
17	*******
18	****
19	****
20	****
21	*****
22	**
23	*****
24	******
25	***********
26	****
27	***********
28	**********
29	*************
30	***
31	******
32	*********
33	****
34	*********
35	*******
36	*****
37	****
38	*****
39	**
40	********
41	**********
42	*******
43	*******
44	*********
45	**********
46	*****
47	********
48	******
49	*********
50	*********
51	************
52	**********
53	*****
54	******
55	******
56	******
57	******
58	****
59	****
60	*******
61	*********
62	*********
63	***
64	*****
65	*******
66	**
67	*****
68	****
69	*****
70	***
71	***
72	*****
73	***
74	****
75	**
76	**
77	******
78	****
79	**
80	***
81	*
82	*
83	******
84	**
85	**
86	*
87	***
88	**
90	**
91	**
92	****
93	**
94	*
96	*
97	****
98	*
99	*
100	**
103	**
105	*
108	*
109	*
110	**
113	*
117	*
118	*
119	*
122	*
139	*
141	*
168	*
174	*
230	*
264	*

Histogram of the intraindividual candidates' ranges (all tests)

1	*
3	*
6	**
7	***
11	*
12	**
20	**
21	**
22	**
25	*
26	***
27	*
28	****
29	**
30	*
31	*
32	**
33	**
34	*
35	**
36	**
38	**
39	*
40	***
41	**
42	***
44	**
45	*
46	******
47	***
48	*
49	*
50	***
51	*
52	**
53	***
54	***
56	***
57	**
58	*
59	**
60	****
62	*
63	**
65	***
66	******
67	*****
68	*
70	*
72	***
73	***
74	**
75	****
76	**
78	**
80	******
81	**
82	**
83	***
85	****
86	********
87	*********
90	****
91	***
92	**
93	******
94	****
95	***
97	**
98	***
99	**
100	******
101	**
103	**
104	**
105	****
106	***
107	***
108	***
109	**
111	**
112	*****
113	****
114	***
115	*
116	*
117	**
119	***
120	*********
122	***
123	**
124	*
125	**
126	**
127	*****
128	*
129	****
130	*
132	*
133	*****
134	*
135	**
136	***
137	**
139	**
140	********
141	**
142	*
143	*
144	**
145	*
146	***
147	******
149	*
150	*
152	**
153	****
154	******
155	**
156	*
158	**
159	*
160	*
161	*
163	*
164	*
165	*
166	*
167	*****
168	*
169	*
170	*
173	**
176	**
177	**
179	****
180	********
182	**
183	***
184	*
185	***
186	**
187	*
188	***
189	**
191	****
192	*
194	*
195	*
197	*
198	**
199	**
200	***
201	*
202	**
204	***
206	**
208	*
210	*
211	*
212	*
215	*
216	**
217	*
218	*
219	*
220	***
223	*
224	**
225	*
226	*
228	*
229	*
230	**
233	*
234	**
236	*
238	**
240	*****
245	*
249	*
250	*
251	*
252	*
254	**
256	*
257	*
259	**
263	**
266	*
268	*
270	*
273	***
276	**
278	*
280	*
286	*
290	*
293	*
294	*
306	*
307	**
310	*
317	**
320	*
321	*
325	*
330	*
332	*
342	*
344	*
352	*
415	*
445	*
753	*