Neural Factorization Machines for Sparse Predictive Analytics
- 7 August 2017
- conference paper
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 355-364
- https://doi.org/10.1145/3077136.3080777
Abstract
Many predictive tasks of web applications need to model categorical variables, such as user IDs and demographics like genders and occupations. To apply standard machine learning techniques, these categorical predictors are always converted to a set of binary features via one-hot encoding, making the resultant feature vector highly sparse. To learn from such sparse data effectively, it is crucial to account for the interactions between features. Factorization Machines (FMs) are a popular solution for efficiently using the second-order feature interactions. However, FM models feature interactions in a linear way, which can be insufficient for capturing the non-linear and complex inherent structure of real-world data. While deep neural networks have recently been applied to learn non-linear feature interactions in industry, such as the Wide&Deep by Google and DeepCross by Microsoft, the deep structure meanwhile makes them difficult to train. In this paper, we propose a novel model Neural Factorization Machine (NFM) for prediction under sparse settings. NFM seamlessly combines the linearity of FM in modelling second-order feature interactions and the non-linearity of neural network in modelling higher-order feature interactions. Conceptually, NFM is more expressive than FM since FM can be seen as a special case of NFM without hidden layers. Empirical results on two regression tasks show that with one hidden layer only, NFM significantly outperforms FM with a 7.3% relative improvement. Compared to the recent deep learning methods Wide&Deep and DeepCross, our NFM uses a shallower structure but offers better performance, being much easier to train and tune in practice.Keywords
This publication has 34 references indexed in Scilit:
- Convex Factorization MachinesPublished by Springer Science and Business Media LLC ,2015
- Learning Hierarchical Representation Model for NextBasket RecommendationPublished by Association for Computing Machinery (ACM) ,2015
- Visual Classification by $\ell _1$ -Hypergraph ModelingIEEE Transactions on Knowledge and Data Engineering, 2015
- CORE: Context-Aware Open Relation Extraction with Factorization MachinesPublished by Association for Computational Linguistics (ACL) ,2015
- Predicting the popularity of web 2.0 items based on user commentsPublished by Association for Computing Machinery (ACM) ,2014
- Attribute-augmented semantic hierarchyPublished by Association for Computing Machinery (ACM) ,2013
- Co-factorization machinesPublished by Association for Computing Machinery (ACM) ,2013
- Factorization Machines with libFMACM Transactions on Intelligent Systems and Technology, 2012
- Fast context-aware recommendations with factorization machinesPublished by Association for Computing Machinery (ACM) ,2011
- Factorization meets the neighborhoodPublished by Association for Computing Machinery (ACM) ,2008