Building high-level features using large scale unsupervised learning
Top Cited Papers
- 1 May 2013
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 8595-8598
- https://doi.org/10.1109/icassp.2013.6639343
Abstract
We consider the problem of building high-level, class-specific feature detectors from only unlabeled data. For example, is it possible to learn a face detector using only unlabeled images? To answer this, we train a deep sparse autoencoder on a large dataset of images (the model has 1 billion connections, the dataset has 10 million 200×200 pixel images downloaded from the Internet). We train this network using model parallelism and asynchronous SGD on a cluster with 1,000 machines (16,000 cores) for three days. Contrary to what appears to be a widely-held intuition, our experimental results reveal that it is possible to train a face detector without having to label images as containing a face or not. Control experiments show that this feature detector is robust not only to translation but also to scaling and out-of-plane rotation. We also find that the same network is sensitive to other high-level concepts such as cat faces and human bodies. Starting from these learned features, we trained our network to recognize 22,000 object categories from ImageNet and achieve a leap of 70% relative improvement over the previous state-of-the-art.Keywords
Other Versions
This publication has 15 references indexed in Scilit:
- Traffic sign recognition with multi-scale Convolutional NetworksPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2011
- High-dimensional signature compression for large-scale image classificationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2011
- Convolutional deep belief networks for scalable unsupervised learning of hierarchical representationsPublished by Association for Computing Machinery (ACM) ,2009
- Why is Real-World Visual Object Recognition Hard?PLoS Computational Biology, 2008
- Reducing the Dimensionality of Data with Neural NetworksScience, 2006
- A Fast Learning Algorithm for Deep Belief NetsNeural Computation, 2006
- Invariant visual representation by single neurons in the human brainNature, 2005
- Slow feature analysis yields a rich repertoire of complex cell propertiesJournal of Vision, 2005
- Aging and the human neocortexExperimental Gerontology, 2003
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998