Sensitivity and Predictive Value of 15 PubMed Search Strategies to Answer Clinical Questions Rated Against Full Systematic Reviews
Open Access
- 12 June 2012
- journal article
- research article
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 14 (3), e85
- https://doi.org/10.2196/jmir.2021
Abstract
Background: Clinicians perform searches in PubMed daily, but retrieving relevant studies is challenging due to the rapid expansion of medical knowledge. Little is known about the performance of search strategies when they are applied to answer specific clinical questions. Objective: To compare the performance of 15 PubMed search strategies in retrieving relevant clinical trials on therapeutic interventions. Methods: We used Cochrane systematic reviews to identify relevant trials for 30 clinical questions. Search terms were extracted from the abstract using a predefined procedure based on the population, interventions, comparison, outcomes (PICO) framework and combined into queries. We tested 15 search strategies that varied in their query (PIC or PICO), use of PubMed’s Clinical Queries therapeutic filters (broad or narrow), search limits, and PubMed links to related articles. We assessed sensitivity (recall) and positive predictive value (precision) of each strategy on the first 2 PubMed pages (40 articles) and on the complete search output. Results: The performance of the search strategies varied widely according to the clinical question. Unfiltered searches and those using the broad filter of Clinical Queries produced large outputs and retrieved few relevant articles within the first 2 pages, resulting in a median sensitivity of only 10%–25%. In contrast, all searches using the narrow filter performed significantly better, with a median sensitivity of about 50% (all P < .001 compared with unfiltered queries) and positive predictive values of 20%–30% (P < .001 compared with unfiltered queries). This benefit was consistent for most clinical questions. Searches based on related articles retrieved about a third of the relevant studies. Conclusions: The Clinical Queries narrow filter, along with well-formulated queries based on the PICO framework, provided the greatest aid in retrieving relevant clinical trials within the 2 first PubMed pages. These results can help clinicians apply effective strategies to answer their questions at the point of care. [J Med Internet Res 2012;14(3):e85]Keywords
This publication has 37 references indexed in Scilit:
- Speed, accuracy, and confidence in Google, Ovid, PubMed, and UpToDate: results of a randomised trialPublished by Oxford University Press (OUP) ,2010
- Evaluating the impact of MEDLINE filters on evidence retrieval: study protocolImplementation Science, 2010
- Diagnostic test systematic reviews: Bibliographic search filters (“Clinical Queries”) for diagnostic accuracy studies perform wellJournal of Clinical Epidemiology, 2009
- Analysis of queries sent to PubMed at the point of care: Observation of search behaviour in a medical teaching hospitalBMC Medical Informatics and Decision Making, 2008
- PubMed related articles: a probabilistic topic-based model for content similarityBMC Bioinformatics, 2007
- Real-time EBM: From Bed Board to Keyboard and BackJournal of General Internal Medicine, 2007
- Information Needs and Information-Seeking Behavior of Primary Care PhysiciansAnnals of Family Medicine, 2007
- Utilization of the PICO framework to improve searching PubMed for clinical questionsBMC Medical Informatics and Decision Making, 2007
- A Day in the Life of PubMed: Analysis of a Typical Day's Query LogJournal of the American Medical Informatics Association, 2007
- Effectiveness of Clinician-selected Electronic Information Resources for Answering Primary Care Physicians' Information NeedsJournal of the American Medical Informatics Association, 2006