Inferential language models for information retrieval
- 1 December 2006
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Asian Language Information Processing
- Vol. 5 (4), 296-322
- https://doi.org/10.1145/1236181.1236183
Abstract
Language modeling (LM) has been widely used in IR in recent years. An important operation in LM is smoothing of the document language model. However, the current smoothing techniques merely redistribute a portion of term probability according to their frequency of occurrences only in the whole document collection. No relationships between terms are considered and no inference is involved. In this article, we propose several inferential language models capable of inference using term relationships. The inference operation is carried out through a semantic smoothing either on the document model or query model, resulting in document or query expansion. The proposed models implement some of the logical inference capabilities proposed in the previous studies on logical models, but with necessary simplifications in order to make them tractable. They are a good compromise between inference power and efficiency. The models have been tested on several TREC collections, both in English and Chinese. It is shown that the integration of term relationships into the language modeling framework can consistently improve the retrieval effectiveness compared with the traditional language models. This study shows that language modeling is a suitable framework to implement basic inference operations in IR effectively.Keywords
This publication has 35 references indexed in Scilit:
- Query expansion and query translation as logical inferenceJournal of the American Society for Information Science and Technology, 2003
- Towards context sensitive information inferenceJournal of the American Society for Information Science and Technology, 2003
- Application of aboutness to functional benchmarking in information retrievalACM Transactions on Information Systems, 2001
- A Logical Model for Information Retrieval based on Propositional Logic and Belief RevisionThe Computer Journal, 2001
- The use of logic in information retrieval modellingThe Knowledge Engineering Review, 1998
- Information retrieval and situation theoryACM SIGIR Forum, 1996
- INFORMATION RETRIEVAL BY LOGICAL IMAGINGJournal of Documentation, 1995
- Introduction to WordNet: An On-line Lexical Database*International Journal of Lexicography, 1990
- Indexing by latent semantic analysisJournal of the American Society for Information Science, 1990
- A non-classical logic for information retrievalThe Computer Journal, 1986