Integrating Prior Knowledge into Deep Learning
- 1 December 2017
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE) in 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA)
- p. 920-923
- https://doi.org/10.1109/icmla.2017.00-37
Abstract
Deep learning allows to develop feature representations and train classification models in a fully integrated way. However, learning deep networks is quite hard and it improves over shallow architectures only if a large number of training data is available. Injecting prior knowledge into the learner is a principled way to reduce the amount of required training data, as the learner does not need to induce the knowledge from the data itself. In this paper we propose a general and principled way to integrate prior knowledge when training deep networks. Semantic Based Regularization (SBR) is used as underlying framework to represent the prior knowledge, expressed as a collection of first-order logic clauses (FOL), and where each task to be learned corresponds to a predicate in the knowledge base. The knowledge base correlates the tasks to be learned and it is translated into a set of constraints which are integrated into the learning process via backpropagation. The experimental results show how the integration of the prior knowledge boosts the accuracy of a state-of-the-art deep network on an image classification task.Keywords
This publication has 8 references indexed in Scilit:
- Learning and reasoning in logic tensor networksPublished by Association for Computing Machinery (ACM) ,2017
- Learning Efficiently in Semantic Based RegularizationPublished by Springer Science and Business Media LLC ,2016
- Bridging logic and kernel machinesMachine Learning, 2011
- Convolutional deep belief networks for scalable unsupervised learning of hierarchical representationsPublished by Association for Computing Machinery (ACM) ,2009
- Learning Deep Architectures for AIFoundations and Trends® in Machine Learning, 2009
- Collective Classification in Network DataAI Magazine, 2008
- Markov logic networksMachine Learning, 2006
- Learning methods for generic object recognition with invariance to pose and lightingPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2004