What, where and who? Classifying events by scene and object recognition
Top Cited Papers
- 1 January 2007
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
Abstract
We propose a first attempt to classify events in static images by integrating scene and object categorizations. We define an event in a static image as a human activity taking place in a specific environment. In this paper, we use a number of sport games such as snow boarding, rock climbing or badminton to demonstrate event classification. Our goal is to classify the event in the image as well as to provide a number of semantic labels to the objects and scene environment within the image. For example, given a rowing scene, our algorithm recognizes the event as rowing by classifying the environment as a lake and recognizing the critical objects in the image as athletes, rowing boat, water, etc. We achieve this integrative and holistic recognition through a generative graphical model. We have assembled a highly challenging database of 8 widely varied sport events. We show that our system is capable of classifying these event classes at 73.4% accuracy. While each component of the model contributes to the final recognition, using scene or objects alone cannot achieve this performance.Keywords
This publication has 17 references indexed in Scilit:
- What do we perceive in a glance of a real-world scene?Journal of Vision, 2007
- Putting Objects in PerspectivePublished by Institute of Electrical and Electronics Engineers (IEEE) ,2006
- Rapid object detection using a boosted cascade of simple featuresPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2005
- Image Parsing: Unifying Segmentation, Detection, and RecognitionInternational Journal of Computer Vision, 2005
- A Semantic Typicality Measure for Natural Scene CategorizationLecture Notes in Computer Science, 2004
- Object class recognition by unsupervised scale-invariant learningPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2003
- Indoor-outdoor image classificationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Object Recognition using Local Affine Frames on Distinguished RegionsPublished by British Machine Vision Association and Society for Pattern Recognition ,2002
- Modeling the Shape of the Scene: A Holistic Representation of the Spatial EnvelopeInternational Journal of Computer Vision, 2001
- Object recognition from local scale-invariant featuresPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1999