Deep Crossing
- 13 August 2016
- conference paper
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 255-262
- https://doi.org/10.1145/2939672.2939704
Abstract
Manually crafted combinatorial features have been the \"secret sauce\" behind many successful models. For web-scale applications, however, the variety and volume of features make these manually crafted features expensive to create, maintain, and deploy. This paper proposes the Deep Crossing model which is a deep neural network that automatically combines features to produce superior models. The input of Deep Crossing is a set of individual features that can be either dense or sparse. The important crossing features are discovered implicitly by the networks, which are comprised of an embedding and stacking layer, as well as a cascade of Residual Units. Deep Crossing is implemented with a modeling tool called the Computational Network Tool Kit (CNTK), powered by a multi-GPU platform. It was able to build, from scratch, two web-scale models for a major paid search engine, and achieve superior results with only a sub-set of the features used in the production models. This demonstrates the potential of using Deep Crossing as a general modeling paradigm to improve existing products, as well as to speed up the development of new models with a fraction of the investment in feature engineering and acquisition of deep domain knowledge.Keywords
This publication has 10 references indexed in Scilit:
- Deep Residual Learning for Image RecognitionPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2016
- Scalable training of deep learning machines by incremental block training with intra-block parallel optimization and blockwise model-update filteringPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2016
- Deep learning in neural networks: An overviewNeural Networks, 2015
- Learning semantic representations using convolutional neural networks for web searchPublished by Association for Computing Machinery (ACM) ,2014
- Machine Learning Paradigms for Speech Recognition: An OverviewIEEE Transactions on Audio, Speech, and Language Processing, 2013
- Learning deep structured semantic models for web search using clickthrough dataPublished by Association for Computing Machinery (ACM) ,2013
- Factorization MachinesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2010
- Internet Advertising and the Generalized Second Price Auction: Selling Billions of Dollars Worth of KeywordsPublished by National Bureau of Economic Research ,2005
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in positionBiological Cybernetics, 1980