The Context of Context Effects

Abstract

The principal purpose of this article is to provide a conceptual framework and heuristic model for considering the existence, magnitude, and consequences of context effects. This purpose is addressed through an extension of some concepts in generalizability theory. In particular, distinctions are drawn between different types of facets and different types of universes. For example, sets of items are distinguished from conditions of measurement typically associated with context effects (e.g., item sequence). In addition, a validity-defining universe of generalization is distinguished from a universe of allowable observations associated with a standardized measurement proce- dure and certain fixed conditions of measurement. Other fixed conditions of measurement are also considered in order to examine context effects involved in applications such as the use of so-called "scrambled" test forms and item or section preequating. It is concluded that context effects are often misunder- stood or masked, that current measurement models have rather serious limitations for examining context effects, and that the importance and magnitude of context effects need to be evaluated in context.