Leveraging the Wisdom of the Crowd for Fine-Grained Recognition
- 1 June 2015
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 38 (4), 666-676
- https://doi.org/10.1109/tpami.2015.2439285
Abstract
Fine-grained recognition concerns categorization at sub-ordinate levels, where the distinction between object classes is highly local. Compared to basic level recognition, fine-grained categorization can be more challenging as there are in general less data and fewer discriminative features. This necessitates the use of a stronger prior for feature selection. In this work, we include humans in the loop to help computers select discriminative features. We introduce a novel online game called “Bubbles” that reveals discriminative features humans use. The player's goal is to identify the category of a heavily blurred image. During the game, the player can choose to reveal full details of circular regions (“bubbles”), with a certain penalty. With proper setup the game generates discriminative bubbles with assured quality. We next propose the “BubbleBank” representation that uses the human selected bubbles to improve machine recognition performance. Finally, we demonstrate how to extend BubbleBank to a view-invariant 3D representation. Experiments demonstrate that our approach yields large improvements over the previous state of the art on challenging benchmarks.Keywords
Funding Information
- Intel
- ISTC
- ONR-MURI (NSF-IIS-1115313)
- Max Planck Center for Visual Computing and Communication
This publication has 37 references indexed in Scilit:
- 3D Object Representations for Fine-Grained CategorizationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2013
- Pose pooling kernels for sub-category recognitionPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2012
- SURFing the point clouds: Selective 3D spatial pyramids for category-level object recognitionPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2012
- Revisiting 3D geometric models for accurate object shape and posePublished by Institute of Electrical and Electronics Engineers (IEEE) ,2011
- Locality-constrained Linear Coding for image classificationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2010
- Multi-view object class detection with a 3D geometric modelPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2010
- What Does Classifying More Than 10,000 Image Categories Tell Us?Lecture Notes in Computer Science, 2010
- Video scene categorization by 3D hierarchical histogram matchingPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2009
- PeekaboomPublished by Association for Computing Machinery (ACM) ,2006
- Labeling images with a computer gamePublished by Association for Computing Machinery (ACM) ,2004