Evaluating Performance of Recommender Systems: An Experimental Comparison
- 1 December 2008
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 1, 735-738
- https://doi.org/10.1109/wiiat.2008.252
Abstract
Much early evaluation work focused specifically on the "accuracy" of recommendation algorithms. Good recommendation (in terms of accuracy) has, however, to be coupled with other considerations. This work suggests measures aiming at evaluating other aspects than accuracy of recommendation algorithms. Other considerations include (1) coverage, which measures the percentage of a data set that a recommender system is able to provide recommendation for, (2) confidence metrics that can help users make more effective decisions, (3) computing time, which measures how quickly an algorithm can produce good recommendations, (4) novelty/serendipity, which measure whether a recommendation is original, and (5) robustness which measure the ability of the algorithm to make good predictions in the presence of noisy or sparse data. Six collaborative recommendation methods are investigated. Results on artificial data sets (for robustness) or on the real MovieLens data set (for accuracy, novelty, and computing time) are included and analyzed, showing that kernel-based algorithms provide the best results overall.Keywords
This publication has 11 references indexed in Scilit:
- Random-Walk Computation of Similarities between Nodes of a Graph with Application to Collaborative RecommendationIEEE Transactions on Knowledge and Data Engineering, 2007
- An Experimental Investigation of Graph Kernels on a Collaborative Recommendation TaskIEEE International Conference on Data Mining (ICDM), 2006
- Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensionsIEEE Transactions on Knowledge and Data Engineering, 2005
- Collaborative recommendationACM Transactions on Internet Technology, 2004
- Evaluating collaborative filtering recommender systemsACM Transactions on Information Systems, 2004
- On the recommending of citations for research papersPublished by Association for Computing Machinery (ACM) ,2002
- The role of transparency in recommender systemsPublished by Association for Computing Machinery (ACM) ,2002
- Recommending and evaluating choices in a virtual community of usePublished by Association for Computing Machinery (ACM) ,1995
- Social information filteringPublished by Association for Computing Machinery (ACM) ,1995
- GroupLensPublished by Association for Computing Machinery (ACM) ,1994