Improving the Reliability of Physician Performance Assessment
- 1 April 2009
- journal article
- Published by Ovid Technologies (Wolters Kluwer Health) in Medical Care
- Vol. 47 (4), 378-387
- https://doi.org/10.1097/mlr.0b013e31818dce07
Abstract
The proliferation of efforts to assess physician performance underscore the need to improve the reliability of physician-level quality measures. Using diabetes care as a model, to address 2 key issues in creating reliable physician-level quality performance scores: estimating the physician effect on quality and creating composite measures. Retrospective longitudinal observational study. A national sample of physicians (n = 210) their patients with diabetes (n = 7574) participating in the National Committee on Quality Assurance-American Diabetes Association's Diabetes Provider Recognition Program. Using 11 diabetes process and intermediate outcome quality measures abstracted from the medical records of participants, we tested each measure for the magnitude of physician-level variation (the physician effect or "thumbprint"). We then combined measures with a substantial physician effect into a composite, physician-level diabetes quality score and tested its reliability. We identified the lowest target values for each outcome measure for which there was a recognizable "physician thumbprint" (ie, intraclass correlation coefficient > or =0.30) to create a composite performance score. The internal consistency reliability (Cronbach's alpha) of the composite score, created by combining the process and outcome measures with an intraclass correlation coefficient > or =0.30, exceeded 0.80. The standard errors of the composite case-mix adjusted score were sufficiently small to discriminate those physicians scoring in the highest from those scoring in the lowest quartiles of the quality of care distribution with no overlap. We conclude that the aggregation of well-tested quality measures that maximize the "physician effect" into a composite measure yields reliable physician-level quality of care scores for patients with diabetes.Keywords
This publication has 50 references indexed in Scilit:
- Disclosure of Individual Surgeon's Performance Rates During Informed ConsentAnnals of Surgery, 2007
- Care Patterns in Medicare and Their Implications for Pay for PerformanceNew England Journal of Medicine, 2007
- Public Reporting and Pay for Performance in Hospital Quality ImprovementNew England Journal of Medicine, 2007
- Pay for Performance in Commercial HMOsNew England Journal of Medicine, 2006
- Public Report Cards — Cardiac Surgery and BeyondNew England Journal of Medicine, 2006
- Paying for Performance — Risks and RecommendationsNew England Journal of Medicine, 2006
- Early Experience With Pay-for-PerformanceJAMA, 2005
- Linking Compensation to Quality — Medicare Payments to PhysiciansNew England Journal of Medicine, 2005
- Sample size considerations in observational health care quality studiesStatistics in Medicine, 2002
- Demonstration of Formulae for True Measurement of CorrelationPublished by University of Illinois Press ,1907