Observational Studies and Randomized Trials

Abstract

In the June 22 issue, Concato et al.¹ compared 5 systematic reviews of randomized, controlled trials and observational studies on the same topic, and Benson and Hartz² evaluated 18 case studies of randomized, controlled trials and observational studies of different interventions and 3 reviews of randomized, controlled trials and observational studies of the same intervention. Do these data constitute a representative sample of all available comparisons in the medical literature? We think not. We know of at least two systematic reviews^3,4 in which observational studies detected effects that were not supported by randomized, controlled trials. It would be counterintuitive if randomization, the most important way to produce groups that are truly comparable with respect to known and unknown prognostic factors at base line, were superfluous for generating valid estimates of effect. Even in trials purported to be randomized, if the randomization is inadequately implemented, higher estimates of effect are produced.^5-7