Project Overview

StyleSnap is an AI-powered feature in the Amazon app that helps users shop by identifying clothing styles in uploaded images and presents users with relevant recommendations.

Organisation

Amazon

Project Context

Deep learning refers to a class of machine learning techniques based on artificial neural networks, which are inspired by the working of the human brain. Neural networks are made up of millions of artificial neurons connected to each other, and can be “trained” to detect images of outfits by feeding it a series of images. For example, if a network is fed thousands of images of maxi and accordion skirts, it will eventually be able to tell the difference between the two styles. If, however, it is presented with one Scottish kilt, it may be confused and predict an incorrect class until enough examples are provided to train it otherwise.

Project Innovation

To get started, all users have to do is click the camera icon in the upper right hand corner of the Amazon App, and select the “StyleSnap” option; then simply upload a photograph or screenshot of a fashion look that they like. StyleSnap will present them with recommendations for similar items on Amazon that match the look in the photo. When providing recommendations, StyleSnap considers a variety of factors such as brand, price range, and customer reviews.

Design Challenge

To have neural networks identify a greater number of classes, a greater number of layers can be stacked on top of each other. The first few layers typically learn concepts such as edges and colors, while the middle layers identify patterns such as “floral” or “denim”. After having passed through all of the layers, the algorithm can accurately identify concepts like fit and outfit style in an image.

One step further, is required, however – feed-forward neural networks will stall and eventually degrade after a certain number of layers have been added. This is known as the vanishing gradient problem, where the signal from the training data is so spread out between layers that it is lost entirely.

Amazon uses residual networks to overcome this problem, as they use shortcuts to allow the training signal to skip over some of the layers in the network. This helps the network learn basic features like “edges” and “patterns” first, and then focus on complex concepts. A unique method developed by Amazon researchers allows the network to learn new concepts while also remembering things it has learned in the past – this is critical for enabling StyleSnap to work through large volumes of data effectively.