The meaning of correlated measurement errors is discussed within a hierarchical framework of error terms provided by true score, first-order factor, and second-order factor models: random error, indicator specific error, and group specific error, respectively. Group specific error can be represented either as extraneous first-order factors or as unwanted components of first-order factors that define a second-order factor. The uncritical use of correlated measurement errors without theoretical justification is shown to lead merely to more acceptable fit while obfuscating a more meaningful theoretical structure.