Learning visual similarity for product design with convolutional neural networks
- 27 July 2015
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 34 (4), 1-10
- https://doi.org/10.1145/2766959
Abstract
Popular sites like Houzz, Pinterest, and LikeThatDecor, have communities of users helping each other answer questions about products in images. In this paper we learn an embedding for visual search in interior design. Our embedding contains two different domains of product images: products cropped from internet scenes, and products in their iconic form. With such a multi-domain embedding, we demonstrate several applications of visual search including identifying products in scenes and finding stylistically similar products. To obtain the embedding, we train a convolutional neural network on pairs of images. We explore several training architectures including re-purposing object classifiers, using siamese networks, and using multitask learning. We evaluate our search quantitatively and qualitatively and demonstrate high quality results for search across multiple visual domains, enabling new applications in interior design.Keywords
Funding Information
- NSF (IIS-1149393, IIS-1011919, IIS-1161645)
This publication has 23 references indexed in Scilit:
- A similarity measure for illustration styleACM Transactions on Graphics, 2014
- Exploratory font selection using crowdsourced attributesACM Transactions on Graphics, 2014
- Scalable Nearest Neighbor Algorithms for High Dimensional DataIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- Return of the Devil in the Details: Delving Deep into Convolutional NetsPublished by British Machine Vision Association and Society for Pattern Recognition ,2014
- Recognizing Image StylePublished by British Machine Vision Association and Society for Pattern Recognition ,2014
- OpenSurfacesACM Transactions on Graphics, 2013
- Micro perceptual human computation for visual tasksACM Transactions on Graphics, 2012
- Large Scale Online Learning of Image Similarity through RankingLecture Notes in Computer Science, 2009
- Fisher Kernels on Visual Vocabularies for Image CategorizationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2007
- Backpropagation Applied to Handwritten Zip Code RecognitionNeural Computation, 1989