Challenges to the validity of topic reconstruction
Open Access
- 1 May 2021
- journal article
- research article
- Published by Springer Science and Business Media LLC in Scientometrics
- Vol. 126 (5), 4511-4536
- https://doi.org/10.1007/s11192-021-03920-3
Abstract
In this paper we utilize an opportunity to construct ground truths for topics in the field of atomic, molecular and optical physics. Our research questions in this paper focus on (i) how to construct a ground truth for topics and (ii) the suitability of common algorithms applied to bibliometric networks to reconstruct these topics. We use the ground truths to test two data models (direct citation and bibliographic coupling) with two algorithms (the Leiden algorithm and the Infomap algorithm). Our results are discomforting: none of the four combinations leads to a consistent reconstruction of the ground truths. No combination of data model and algorithm simultaneously reconstructs all micro-level topics at any resolution level. Meso-level topics are not reconstructed at all. This suggests (a) that we are currently unable to predict which combination of data model, algorithm and parameter setting will adequately reconstruct which (types of) topics, and (b) that a combination of several data models, algorithms and parameter settings appears to be necessary to reconstruct all or most topics in a set of papers.Funding Information
- Bundesministerium für Bildung und Forschung (01PU17003)
- Technische Universität Berlin
This publication has 40 references indexed in Scilit:
- A new methodology for constructing a publication‐level classification system of scienceJournal of the American Society for Information Science and Technology, 2012
- Compression of Flow Can Reveal Overlapping-Module Organization in NetworksPhysical Review X, 2011
- Co‐citation analysis, bibliographic coupling, and direct citation: Which citation approach represents the research front most accurately?Journal of the American Society for Information Science and Technology, 2010
- Using global mapping to create more accurate document-level maps of research fieldsJournal of the American Society for Information Science and Technology, 2010
- Comparative study on methods of detecting research fronts using different types of citationJournal of the American Society for Information Science and Technology, 2008
- Maps of random walks on complex networks reveal community structureProceedings of the National Academy of Sciences of the United States of America, 2008
- Genesis and Development of a Scientific Fact.Contemporary Sociology: A Journal of Reviews, 1982
- Central Problems in Social Theory: Action, Structure and Contradiction in Social Analysis.Philosophy and Phenomenological Research, 1980
- State of the Field The Conceptualization of Scientific SpecialtiesThe Sociological Quarterly, 1976
- The Phenomenology of the Social World.American Sociological Review, 1968