Messages as replications: Toward a message‐centered design strategy

Abstract

Three significant points of controversy separate Hunter, Hamilton, and Allen's defense of single‐message designs from our suggestion that messages be replicated within experiments. We respond to each of these controversies. First, we examine their claim that “controlled”; single‐message designs completely eliminate all confounding of manipulated variables with other possible influences on the dependent variable and show it to rest on manifestly implausible assumptions. Second, we show why researchers should plan for the possibility of nonuniform treatment effects across messages, and so use multiple‐message designs; contrary to Hunter et al. ‘s suggestion, the available empirical evidence shows that treatment effects can and do vary across messages. Third, we discuss the advantages of having message replications both within and between studies, as opposed to Hunter et al.’s suggestion that such replication occur only between studies; multiple‐message designs provide greater reliability in estimation of treatment effects, equivalent power for detection of variability in treatment effects, and easier identification of moderator variables. Other issues raised by Hunter, Hamilton, and Allen (nested vs. crossed designs, the desirability of experimental manipulation of messages, the benefits of meta‐analysis) are in fact not controversial.