Learning on knowledge graph dynamics provides an early warning of impactful research
- 17 May 2021
- journal article
- research article
- Published by Springer Science and Business Media LLC in Nature Biotechnology
- Vol. 39 (10), 1300-1307
- https://doi.org/10.1038/s41587-021-00907-6
Abstract
The scientific ecosystem relies on citation-based metrics that provide only imperfect, inconsistent and easily manipulated measures of research quality. Here we describe DELPHI (Dynamic Early-warning by Learning to Predict High Impact), a framework that provides an early-warning signal for ‘impactful’ research by autonomously learning high-dimensional relationships among features calculated across time from the scientific literature. We prototype this framework and deduce its performance and scaling properties on time-structured publication graphs from 1980 to 2019 drawn from 42 biotechnology-related journals, including over 7.8 million individual nodes, 201 million relationships and 3.8 billion calculated metrics. We demonstrate the framework’s performance by correctly identifying 19/20 seminal biotechnologies from 1980 to 2014 via a blinded retrospective study and provide 50 research papers from 2018 that DELPHI predicts will be in the top 5% of time-rescaled node centrality in the future. We propose DELPHI as a tool to aid in the construction of diversified, impact-optimized funding portfolios.Keywords
Funding Information
- MIT Media Lab, MIT Center for Bits and Atoms
This publication has 29 references indexed in Scilit:
- The measure of research meritScience, 2014
- Conform and be fundedNature, 2012
- Predicting scientific successNature, 2012
- We Have Met the Enemy…and He is Us: Lessons from Twenty Years of the Kauffman Foundation's Investments in Venture Capital Funds and the Triumph of Hope Over ExperienceSSRN Electronic Journal, 2012
- Coercive Citation in Academic PublishingScience, 2012
- Local bias in venture capital investmentsJournal of Empirical Finance, 2010
- Long-Term Propagation of Distinct Hematopoietic Differentiation Programs In VivoCell Stem Cell, 2007
- Not-so-deep impactNature, 2005
- G9a histone methyltransferase plays a dominant role in euchromatic histone H3 lysine 9 methylation and is essential for early embryogenesisGenes & Development, 2002
- Why the impact factor of journals should not be used for evaluating researchBMJ, 1997