Recipe recognition with large multimodal food dataset
- 1 June 2015
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
Abstract
This paper deals with automatic systems for image recipe recognition. For this purpose, we compare and evaluate leading vision-based and text-based technologies on a new very large multimodal dataset (UPMC Food-101) containing about 100,000 recipes for a total of 101 food categories. Each item in this dataset is represented by one image plus textual information. We present deep experiments of recipe recognition on our dataset using visual, textual information and fusion. Additionally, we present experiments with text-based embedding technology to represent any food word in a semantical continuous space. We also compare our dataset features with a twin dataset provided by ETHZ university: we revisit their data collection protocols and carry out transfer learning schemes to highlight similarities and differences between both datasets. Finally, we propose a real application for daily users to identify recipes. This application is a web search engine that allows any mobile device to send a query image and retrieve the most relevant recipes in our dataset.Keywords
This publication has 12 references indexed in Scilit:
- Food image recognition with deep convolutional featuresPublished by Association for Computing Machinery (ACM) ,2014
- Comparative Study of the Routine Daily Usability of FoodLogJournal of Diabetes Science and Technology, 2014
- Food Balance Estimation by Using Personal Dietary Tendencies in a Multimedia Food LogIEEE Transactions on Multimedia, 2013
- Pooling in image representation: The visual codeword point of viewComputer Vision and Image Understanding, 2013
- A database for fine grained activity detection of cooking activitiesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2012
- SALSAS: Sub-linear active learning strategy with approximate k-NN searchPattern Recognition, 2011
- An Overview of the Technology Assisted Dietary Assessment Project at Purdue UniversityPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2010
- Harvesting Image Databases from the WebIEEE Transactions on Pattern Analysis and Machine Intelligence, 2010
- PFID: Pittsburgh fast-food image datasetPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2009
- Active Learning Methods for Interactive Image RetrievalIEEE Transactions on Image Processing, 2008