Recurrent Coupled Topic Modeling over Sequential Documents
- 20 July 2021
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Knowledge Discovery From Data
- Vol. 16 (1), 1-32
- https://doi.org/10.1145/3451530
Abstract
The abundant sequential documents such as online archival, social media, and news feeds are streamingly updated, where each chunk of documents is incorporated with smoothly evolving yet dependent topics. Such digital texts have attracted extensive research on dynamic topic modeling to infer hidden evolving topics and their temporal dependencies. However, most of the existing approaches focus on single-topic-thread evolution and ignore the fact that a current topic may be coupled with multiple relevant prior topics. In addition, these approaches also incur the intractable inference problem when inferring latent parameters, resulting in a high computational cost and performance degradation. In this work, we assume that a current topic evolves from all prior topics with corresponding coupling weights, forming the multi-topic-thread evolution. Our method models the dependencies between evolving topics and thoroughly encodes their complex multi-couplings across time steps. To conquer the intractable inference challenge, a new solution with a set of novel data augmentation techniques is proposed, which successfully discomposes the multi-couplings between evolving topics. A fully conjugate model is thus obtained to guarantee the effectiveness and efficiency of the inference technique. A novel Gibbs sampler with a backward–forward filter algorithm efficiently learns latent time-evolving parameters in a closed-form. In addition, the latent Indian Buffet Process compound distribution is exploited to automatically infer the overall topic number and customize the sparse topic proportions for each sequential document without bias. The proposed method is evaluated on both synthetic and real-world datasets against the competitive baselines, demonstrating its superiority over the baselines in terms of the low per-word perplexity, high coherent topics, and better document time prediction.Keywords
Funding Information
- the National Key D&R Program of China (2019YFB1600704)
- FDCT (FDCT/0045/2019/A1, FDCT/0007/2018/A1)
- GSTIC (EF005/FST-GZG/2019/GSTIC)
- University of Macau (MYRG2018-00129-FST)
- GDST (2019B111106001)
This publication has 29 references indexed in Scilit:
- Dirichlet-Hawkes Processes with Applications to Clustering Continuous-Time Document StreamsPublished by Association for Computing Machinery (ACM) ,2015
- Coupling learning of complex interactionsInformation Processing & Management, 2015
- Exploring the Space of Topic Coherence MeasuresPublished by Association for Computing Machinery (ACM) ,2015
- Coupled term-term relation analysis for document clusteringPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2013
- A biterm topic model for short textsPublished by Association for Computing Machinery (ACM) ,2013
- TM-LDAPublished by Association for Computing Machinery (ACM) ,2012
- Online multiscale dynamic topic modelsPublished by Association for Computing Machinery (ACM) ,2010
- Multiscale topic tomographyPublished by Association for Computing Machinery (ACM) ,2007
- Hierarchical Dirichlet ProcessesJournal of the American Statistical Association, 2006
- Finding scientific topicsProceedings of the National Academy of Sciences of the United States of America, 2004