Do Category Rating Scales Produce Biased Preference Weights for a Health Index?

Abstract

Category rating scales are frequently used to measure social preferences in health decision models. The use of category rating scale methods has been questioned because subjects allegedly attempt to use rating scale categories equally often. This results in scale values that are contingent upon the context in which a stimulus is presented. In Experiment 1, subjects were randomly divided into five groups that rated a series of health case descriptions using the category method. Extensive pretesting had established the scale values for the stimuli. For one group, all values of the stimuli were evenly distributed across the response scale (as determined by pretests). For a second group, only items from the top of the scale were used; for a third group, only items with low values were used; for the fourth group items with low and high values (but not middle range) were presented; and for the fifth, items with medium values were presented. The results demonstrated that the values for items remain essentially the same, regardless of the group in which they are presented. These results were confirmed and extended in a second experiment. Experiment 3 demonstrated that subjects will attempt to use categories equally often when the stimuli are not health case descriptions but only when the response continuum and end-points of the rating scale are poorly defined. It is suggested that rating scales can provide valid and reliable results if the response continuum is made clear to subjects.