Feasibility of Real-Time Satisfaction Surveys Through Automated Analysis of Patients' Unstructured Comments and Sentiments
- 1 January 2012
- journal article
- Published by Ovid Technologies (Wolters Kluwer Health) in Quality Management in Health Care
- Vol. 21 (1), 9-19
- https://doi.org/10.1097/qmh.0b013e3182417fc4
Abstract
This article shows how sentiment analysis (an artificial intelligence procedure that classifies opinions expressed within the text) can be used to design real-time satisfaction surveys. To improve participation, real-time surveys must be radically short. The shortest possible survey is a comment card. Patients' comments can be found online at sites organized for rating clinical care, within e-mails, in hospital complaint registries, or through simplified satisfaction surveys such as "Minute Survey." Sentiment analysis uses patterns among words to classify a comment into a complaint, or praise. It further classifies complaints into specific reasons for dissatisfaction, similar to broad categories found in longer surveys such as Consumer Assessment of Healthcare Providers and Systems. In this manner, sentiment analysis allows one to re-create responses to longer satisfaction surveys from a list of comments. To demonstrate, this article provides an analysis of sentiments expressed in 995 online comments made at the RateMDs.com Web site. We focused on pediatrician and obstetrician/gynecologist physicians in District of Columbia, Maryland, and Virginia. We were able to classify patients' reasons for dissatisfaction and the analysis provided information on how practices can improve their care. This article reports the accuracy of classifications of comments. Accuracy will improve as the number of comments received increases. In addition, we ranked physicians using the concept of time-to-next complaint. A time-between control chart was used to assess whether time-to-next complaint exceeded historical patterns and therefore suggested a departure from norms. These findings suggest that (1) patients' comments are easily available, (2) sentiment analysis can classify these comments into complaints/praise, and (3) time-to-next complaint can turn these classifications into numerical benchmarks that can trace impact of improvements over time. The procedures described in the article show that real-time satisfaction surveys are possible.Keywords
This publication has 9 references indexed in Scilit:
- Patients’ Evaluations of Health Care Providers in the Era of Social Networking: An Analysis of Physician-Rating WebsitesJournal of General Internal Medicine, 2010
- Attributing Sources of Variation in Patients’ Experiences of Ambulatory CareMedical Care, 2009
- Will doctor rating sites improve standards of care? YesBMJ, 2009
- Will doctor rating sites improve the quality of care? NoBMJ, 2009
- Rethinking satisfaction surveys: time to next complaint.The Joint Commission Journal on Quality and Patient Safety, 2009
- Rethinking Satisfaction SurveysQuality Management in Health Care, 2008
- Practice Features Associated With Patient-Reported Accessibility, Continuity, and Coordination of Primary Health CareAnnals of Family Medicine, 2008
- Psychological Research Online: Report of Board of Scientific Affairs' Advisory Group on the Conduct of Research on the Internet.American Psychologist, 2004
- Health Care QualityJAMA, 1997