The order effect: reflections on the validity of multiple test presentations

Abstract

SYNOPSIS The effect of order has been previously addressed in test–retest studies but its effect on the presentation of more than one test, in the same time, is often underestimated. This study has shown that when two rating scales were given sequentially, each scored lower if received second in the pair, and that this particular form of order effect (occasion effect) was enough to affect markedly agreement on caseness when utilizing standard cut-off scores. It is therefore important that research workers not only adopt design strategies such as counterbalanced presentation, but also analyse their data for the effects of test order and position within such designs.