The Evaluation of Large Research Initiatives

Abstract

Over the past few decades there has been a rise in the number of federally funded large scientific research initiatives, with increased calls to evaluate their processes and outcomes. This article describes efforts to evaluate such initiatives in one agency within the U.S. federal government. The authors introduce the Evaluation of Large Initiatives (ELI) project, a preliminary effort to explore how to accomplish such evaluation. They describe a pilot effort of this project to evaluate the Transdisciplinary Tobacco Use Research Center (TTURC) initiative of the National Cancer Institute. They present a summary of this pilot evaluation including the methods used (concept mapping, logic modeling, a detailed researcher survey, content analysis and systematic peer-evaluation of progress reports, bibliometric analysis and peer evaluation of publications and citations, and financial expenditures analysis) and a brief overview of results. Finally, they discuss several important lessons and recommendations that emerged from this work.