Correlation of attending and patient assessment of resident communication skills in the emergency department

Abstract

Background Communication and interpersonal skills are one of the Accreditation Council for Graduate Medical Education’s six core competencies. Validated methods for assessing these among trainees are lacking. Educators have developed various communication assessment tools from both the supervising attending and patient perspectives. How these different assessment methods and tools compare with each other remains unknown. The goal of this study was to determine the degree of agreement between attending and patient assessment of resident communication skills. Methods This was a retrospective study of Emergency Medicine (EM) residents at an academic medical center. From July 2017 – June 2018, residents were assessed on communication skills during their emergency department shifts by both their supervising attending physicians and their patients. The attendings rated residents’ communication skills with patients, colleagues, and nursing/ancillary staff using a 1-5 Likert scale. Patients completed the modified Communication Assessment Tool (CAT), a 14-item questionnaire based on a 1-5 Likert scale. Mean attending ratings and patient CAT scores were calculated for each resident. Means were divided into tertiles due to nonparametric distribution of scores. Agreement between attending and patient ratings of residents were measured using Cohen’s kappa for each attending evaluation question. Scores were weighted to assign adjacent tertiles partial agreement. Results During the study period, 1097 attending evaluations and 952 patient evaluations were completed for 26 residents. Attending scores and CAT scores of the residents showed slight to fair agreement in the following three domains: patient communication (k 0.21), communication with colleagues (k 0.21) and communication with nursing/ancillary staff (k 0.26). Conclusions Attending and patient ratings of EM residents’ communication skills show slight to fair agreement. The use of different types of raters may be beneficial in fully assessing trainees’ communication skills.