Outcomes of Endovascular Treatments of Aneurysms: Observer Variability and Implications for Interpreting Case Series and Planning Randomized Trials
Open Access
- 22 December 2011
- journal article
- research article
- Published by American Society of Neuroradiology (ASNR) in American Journal of Neuroradiology
- Vol. 33 (4), 626-631
- https://doi.org/10.3174/ajnr.a2848
Abstract
BACKGROUND AND PURPOSE: Angiographic results are commonly used as a surrogate marker of success of coiling of intracranial aneurysms. Inter- and intraobserver agreement in judging angiographic results remain poorly characterized. Our goal was to offer such an evaluation of a grading scale commonly used to evaluate results of endovascular treatment of aneurysms. MATERIALS AND METHODS: A portfolio of 90 angiographic images from 45 patients selected from the core lab data base of a randomized trial was sent to 12 observers on 2 occasions more than 3 months apart. The variability of a 3-value grading scale used to score angiographic results and of a final judgment regarding the presence of a recurrence was studied using κ statistics. RESULTS: Ten participants responded once and 6 responded twice. Agreement was poor to moderate (κ = 0.28–0.5) for senior and junior observers judging angiographic results immediately or 12–18 months after treatment. Agreement reached a reassuring “substantial” (κ = 0.62) level, with a dichotomous presence-absence of a major recurrence, and intraobserver agreement was better in experienced core lab assessors. CONCLUSIONS: There is an important variability in the assessment of angiographic outcomes of endovascular treatments, rendering comparisons between publications risky, if not invalid. A simple dichotomous judgment can be used as a surrogate outcome in randomized trials designed to assess the value of new endovascular devices.This publication has 23 references indexed in Scilit:
- The design of the STenting in Aneurysm Treatments (STAT) trialJournal of NeuroInterventional Surgery, 2011
- Hydrogel-coated coils versus bare platinum coils for the endovascular treatment of intracranial aneurysms (HELPS): a randomised controlled trialThe Lancet, 2011
- Consensus Interpretation in Imaging Research: Is There a Better Way?Radiology, 2010
- A Randomized Trial Comparing Platinum and Hydrogel-Coated Coils in Patients Prone to Recurrence after Endovascular Treatment (The PRET Trial)Interventional Neuroradiology, 2008
- Retreatment of Ruptured Cerebral Aneurysms in Patients Randomized by Coiling or Clipping in the International Subarachnoid Aneurysm Trial (ISAT)Stroke, 2007
- Intracranial Aneurysms: Treatment with Bare Platinum Coils—Aneurysm Packing, Complex Coils, and Angiographic RecurrenceRadiology, 2007
- Long-Term Angiographic Recurrences After Selective Endovascular Treatment of Aneurysms With Detachable CoilsStroke, 2003
- Misguided efforts and future challenges for research on "diagnostic tests"Journal of Epidemiology and Community Health, 2002
- Endovascular Treatment of Unruptured AneurysmsStroke, 2001
- High agreement but low Kappa: I. the problems of two paradoxesJournal of Clinical Epidemiology, 1990