Deep-based Ingredient Recognition for Cooking Recipe Retrieval
Top Cited Papers
- 1 October 2016
- conference paper
- conference paper
- Published by Association for Computing Machinery (ACM)
Abstract
Retrieving recipes corresponding to given dish pictures facilitates the estimation of nutrition facts, which is crucial to various health relevant applications. The current approaches mostly focus on recognition of food category based on global dish appearance without explicit analysis of ingredient composition. Such approaches are incapable for retrieval of recipes with unknown food categories, a problem referred to as zero-shot retrieval. On the other hand, content-based retrieval without knowledge of food categories is also difficult to attain satisfactory performance due to large visual variations in food appearance and ingredient composition. As the number of ingredients is far less than food categories, understanding ingredients underlying dishes in principle is more scalable than recognizing every food category and thus is suitable for zero-shot retrieval. Nevertheless, ingredient recognition is a task far harder than food categorization, and this seriously challenges the feasibility of relying on them for retrieval. This paper proposes deep architectures for simultaneous learning of ingredient recognition and food categorization, by exploiting the mutual but also fuzzy relationship between them. The learnt deep features and semantic labels of ingredients are then innovatively applied for zero-shot retrieval of recipes. By experimenting on a large Chinese food dataset with images of highly complex dish appearance, this paper demonstrates the feasibility of ingredient recognition and sheds light on this zero-shot problem peculiar to cooking recipe retrieval.Keywords
Funding Information
- National Hi-Tech Research and Development Program(863 Program) of China (2014AA015102)
- National Natural Science Foundation of China (61272290)
This publication has 26 references indexed in Scilit:
- Tastes and Textures Estimation of Foods Based on the Analysis of Its Ingredients List and ImagePublished by Springer Science and Business Media LLC ,2015
- FoodLog: Multimedia Tool for Healthcare ApplicationsIEEE MultiMedia, 2015
- Food image recognition using deep convolutional network with pre-training and fine-tuningPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- An adaptive detecting strategy against motion vector-based steganographyPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Recipe recognition with large multimodal food datasetPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Leveraging Context to Support Automated Food Recognition in RestaurantsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Deep Learning Face Representation from Predicting 10,000 ClassesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2014
- Automatic Chinese food identification and quantity estimationPublished by Association for Computing Machinery (ACM) ,2012
- Histograms of Oriented Gradients for Human DetectionPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2005
- Similarity of color imagesPublished by SPIE-Intl Soc Optical Eng ,1995