Web-Scale Multimedia Analysis: Does Content Matter?

Abstract

The initial success of Web-image search was based exclusively on the text around an image. Certainly we have progressed since then. But recent research results dramatically beg to differ. For example, if you want to judge the similarity of two different pieces of music, should you look at the musical notes, or should you look at what people say about the music? Similarly, how should you find the best movie to recommend to a friend? Shouldn't the genre of the movie matter? Or when tagging a photo, is it better to look at the pixels, or where the picture lives on the Internet? I want to think that content matters, but in all three cases, metadata about the content proves to be more useful. It's useful to look at several examples where content has lost out to other forms of data. These examples come from the worlds of music, movies, and images.