Item-rest correlation
© Paul Cooijmans
Explanation
In item analysis, item-rest correlation is an indicator of the validity or discriminative power of an item. It is computed as the correlation between the item's scores and the combined score over the remaining items of the test. In practice one may compute this as the correlation with [total score minus present item score].
In general, the idea is that when item-rest correlation is near zero or negative, the item is not contributing to what the test is measuring and one should investigate whether the item should be removed or revised. However, several points need to be taken into account when interpreting an item-rest correlation:
- Since items are mostly scored dichotomously (0 or 1), item variance is small and therefore item-rest correlations tend to be lower than correlations between two non-dichotomous variables.
- Multiple-choice items with few answering options (like around six or less) will have lower item-rest correlations than open items due to noise from guessing.
- If q (item hardness) is very high or very low for a given item, its item-rest correlation is likely not significant due to the very small variance of such an item (a very large number of test submissions would be needed to make such a correlation significant).
- Due to these several causes that depress item-rest correlations, fairly many test submissions are needed to make this form of item analysis significant (around 60 submissions is no luxury).
- It may happen that item-rest correlation is near zero but one can not find anything substantial wrong with the item. One will then have to decide whether to keep the item based on non-statistical confidence in the item's validity, or to remove it based on a strictly statistical approach to test construction.
- Factors like answer leakage and fraud may actually increase the inter-item correlations, so one should be aware that high item-rest correlations do no imply that all is well with the test.
