Exploring the Behavior of Hospital Composite Performance Measures
- 18 December 2007
- journal article
- research article
- Published by Ovid Technologies (Wolters Kluwer Health) in Circulation
- Vol. 116 (25), 2969-2975
- https://doi.org/10.1161/circulationaha.107.703553
Abstract
Background— Composite scores that combine several performance measures into a single ranking are becoming the accepted metric for assessing hospital performance. In particular, the Centers for Medicare & Medicaid Services Hospital Quality Incentive Demonstration (HQID) project bases financial rewards and penalties on these scores. Although the HQID composite calculation is straightforward and easily understood, its method of combining process and outcome measures has not been validated. Methods and Results— Using data on 530 hospitals from the Society of Thoracic Surgeons National Cardiac Database, we replicated the HQID methodology with 6 nationally endorsed performance measures (5 process measures plus survival) for coronary artery bypass surgery. Composite scores were essentially determined by process measure performance alone; the survival component explained only 4% of the composite score’s total variance. This result persisted even when the survival component was allowed a 5-fold greater weighting in the composite summary. The popular “all-or-none” measurement approach was also dominated by the process component. Substantial disagreement was found among hospital rankings when several alternative methods were used; up to 60% of hospitals eligible for the top financial reward under HQID would change designation depending on the composite methodology used. The application of a simple statistical adjustment (standardization) to each method would provide more consistent results and a more balanced assessment of performance based on both process and outcomes. Conclusions— Existing methods used to create composite performance measures have remarkably different weighting of process versus outcomes metrics and lead to highly divergent provider rankings. Simple alternative methods can create more balanced process-outcome performance assessments.Keywords
This publication has 9 references indexed in Scilit:
- Quality Measurement in Adult Cardiac Surgery: Part 2—Statistical Considerations in Composite Measure Scoring and Provider RatingThe Annals of Thoracic Surgery, 2007
- Measuring Performance For Treating Heart Attacks And Heart Failure: The Case For Outcomes MeasurementHealth Affairs, 2007
- All-or-None Measurement Raises the Bar on PerformanceJAMA, 2006
- Are unaudited records from an outcomes registry database accurate?The Annals of Thoracic Surgery, 2004
- Validity of the Society of Thoracic Surgeons National Adult Cardiac Surgery DatabaseThe Annals of Thoracic Surgery, 2004
- Procedural Volume as a Marker of Quality for CABG SurgeryJAMA, 2004
- Patterns of Secondary Prevention in Older Patients Undergoing Coronary Artery Bypass Grafting During Hospitalization for Acute Myocardial InfarctionCirculation, 2003
- The society of thoracic surgeons: 30-day operative mortality and morbidity risk modelsThe Annals of Thoracic Surgery, 2003
- Preoperative β-Blocker Use and Mortality and Morbidity Following CABG Surgery in North AmericaPublished by American Medical Association (AMA) ,2002