Given a photo of a chair, lamp, or some other item, a new service will tell you who makes it and where to buy it, and show you pictures of how it might look in various rooms.
“It seems a lot of people want to buy things they see in someone else’s home or in a photo, but they don’t know where to look,” says Sean Bell, a doctoral candidate in computer science at Cornell University. Bell and Kavita Bala, professor of computer science, describe their method for “learning visual similarity for product design” in a paper published in ACM Transactions on Graphics.
The system relies on “deep learning,” a neural network that lets a computer match a submitted photo with a vast database of “iconic images” from manufacturers’ catalogs or specialized websites devoted to home furnishings.
A neural network is a computer program inspired by the working of neurons in the human brain. As data is passed through the network, locations in memory that are activated repeatedly are increased in value, just as a biological brain forms synapses. “Deep learning” combines several layers of neurons that represent different aspects of the data—earlier layers typically represent edges and lines, middle layers represent parts and shapes, and later layers represent entire objects and concepts.
The researchers used crowdsourcing to prepare a collection of images to train the neural network. On the Amazon Mechanical Turk service, where home workers can earn a few cents at a time to perform simple microtasks, they showed workers scene photos and asked them to draw boxes around objects. The resulting collection of boxes, along with their matching iconic images, was used to train the computer.
People using the service won’t want to wait for results. Rather than force the computer to go through the entire database looking for a match, the system begins by using the neural network to generate a “fingerprint” of a submitted image, based on very broad characteristics of how the pixels are arranged. Then the computer can search just a local area of the database, analogous to searching for a phone number in just one area code.
Bala and Bell have formed a startup company, GrokStyle, to offer the service on a subscription basis to retailers and design professionals, with support from the Small Business Innovation Research Program through which the National Science Foundation and other government agencies encourage the commercialization of new technology to advance the economy.
“I’m excited by the importance of this for the design industry,” says Bala. In the future, the researchers says, similar systems might be developed for other kinds of products, such as clothing.
Source: Cornell University