Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations
Top Cited Papers
- 14 June 2009
- conference paper
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 609-616
- https://doi.org/10.1145/1553374.1553453
Abstract
There has been much interest in unsupervised learning of hierarchical generative models such as deep belief networks. Scaling such models to full-sized, high-dimensional images remains a difficult problem. To address this problem, we present the convolutional deep belief network, a hierarchical generative model which scales to realistic image sizes. This model is translation-invariant and supports efficient bottom-up and top-down probabilistic inference. Key to our approach is probabilistic max-pooling, a novel technique which shrinks the representations of higher layers in a probabilistically sound way. Our experiments show that the algorithm learns useful high-level visual features, such as object parts, from unlabeled images of objects and natural scenes. We demonstrate excellent performance on several visual recognition tasks and show that our model can perform hierarchical (bottom-up and top-down) inference over full-sized images.Keywords
Funding Information
- Defense Advanced Research Projects Agency (FA8750-05-2-0249)
This publication has 17 references indexed in Scilit:
- Unsupervised Learning of Invariant Feature Hierarchies with Applications to Object RecognitionPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2007
- Reducing the Dimensionality of Data with Neural NetworksScience, 2006
- Multiclass Object Recognition with Sparse, Localized FeaturesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2006
- A Fast Learning Algorithm for Deep Belief NetsNeural Computation, 2006
- Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene CategoriesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2006
- Representation of Angles Embedded within Contour Stimuli in Area V2 of Macaque MonkeysJournal of Neuroscience, 2004
- Hierarchical Bayesian inference in the visual cortexJournal of the Optical Society of America A, 2003
- Training Products of Experts by Minimizing Contrastive DivergenceNeural Computation, 2002
- Emergence of simple-cell receptive field properties by learning a sparse code for natural imagesNature, 1996
- Backpropagation Applied to Handwritten Zip Code RecognitionNeural Computation, 1989