Self-taught hashing for fast similarity search
- 19 July 2010
- conference paper
- conference paper
- Published by Association for Computing Machinery (ACM)
Abstract
The ability of fast similarity search at large scale is of great importance to many Information Retrieval (IR) applications. A promising way to accelerate similarity search is semantic hashing which designs compact binary codes for a large number of documents so that semantically similar documents are mapped to similar codes (within a short Hamming distance). Although some recently proposed techniques are able to generate high-quality codes for documents known in advance, obtaining the codes for previously unseen documents remains to be a very challenging problem. In this paper, we emphasise this issue and propose a novel Self-Taught Hashing (STH) approach to semantic hashing: we first find the optimal l-bit binary codes for all documents in the given corpus via unsupervised learning, and then train l classifiers via supervised learning to predict the l-bit code for any query document unseen before. Our experiments on three real-world text datasets show that the proposed approach using binarised Laplacian Eigenmap (LapEig) and linear Support Vector Machine (SVM) outperforms state-of-the-art techniques significantlyKeywords
This publication has 31 references indexed in Scilit:
- Learning to hash: forgiving hash functions and applicationsData Mining and Knowledge Discovery, 2008
- Scene completion using millions of photographsACM Transactions on Graphics, 2007
- Reducing the Dimensionality of Data with Neural NetworksScience, 2006
- A Fast Learning Algorithm for Deep Belief NetsNeural Computation, 2006
- Content-based multimedia information retrievalACM Transactions on Multimedia Computing, Communications, and Applications, 2006
- Laplacian Eigenmaps for Dimensionality Reduction and Data RepresentationNeural Computation, 2003
- Machine learning in automated text categorizationACM Computing Surveys, 2002
- Normalized cuts and image segmentationIeee Transactions On Pattern Analysis and Machine Intelligence, 2000
- New spectral methods for ratio cut partitioning and clusteringIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 1992
- A technique for counting ones in a binary computerCommunications of the ACM, 1960