Research Note—The Halo Effect in Multicomponent Ratings and Its Implications for Recommender Systems: The Case of Yahoo! Movies
- 1 March 2012
- journal article
- Published by Institute for Operations Research and the Management Sciences (INFORMS) in Information Systems Research
- Vol. 23 (1), 231-246
- https://doi.org/10.1287/isre.1100.0336
Abstract
Collaborative filtering algorithms learn from the ratings of a group of users on a set of items to find personalized recommendations for each user. Traditionally they have been designed to work with one-dimensional ratings. With interest growing in recommendations based on multiple aspects of items, we present an algorithm for using multicomponent rating data. The presented mixture model-based algorithm uses the component rating dependency structure discovered by a structure learning algorithm. The structure is supported by the psychometric literature on the halo effect. This algorithm is compared with a set of model-based and instance-based algorithms for single-component ratings and their variations for multicomponent ratings. We evaluate the algorithms using data from Yahoo! Movies. Use of multiple components leads to significant improvements in recommendations. However, we find that the choice of algorithm depends on the sparsity of the training data. It also depends on whether the task of the algorithm is to accurately predict ratings or to retrieve relevant items. In our experiments a model-based multicomponent rating algorithm is able to better retrieve items when training data are sparse. However, if the training data are not sparse, or if we are trying to predict the rating values accurately, then the instance-based multicomponent rating collaborative filtering algorithms perform better. Beyond generating recommendations we show that the proposed model can fill in missing rating components. Theories in psychometric literature and the empirical evidence suggest that rating specific aspects of a subject is difficult. Hence, filling in the missing component values leads to the possibility of a rater support system to facilitate gathering of multicomponent ratings.Keywords
This publication has 30 references indexed in Scilit:
- Does true halo affect observed halo?Journal of Applied Psychology, 1988
- A reexamination of the relation between halo error and accuracy.Journal of Applied Psychology, 1988
- Ubiquitous halo.Psychological Bulletin, 1981
- Statistical control of halo error in performance ratings.Journal of Applied Psychology, 1980
- Discriminability in Multidimensional Performance EvaluationsApplied Psychological Measurement, 1979
- INFLUENCE OF IRRELEVANT CUES AND ALTERNATE FORMS OF GRAPHIC RATING SCALES ON THE HALO EFFECT1Personnel Psychology, 1977
- Training managers to minimize rating errors in the observation of behavior.Journal of Applied Psychology, 1975
- How relevant is an individual difference theory of personality?1Journal of Personality, 1975
- Removing halo from job evaluation factor structure.Journal of Applied Psychology, 1965
- Some characteristics of intrajudge trait intercorrelations.Psychological Monographs: General and Applied, 1962