Let’s start with a definition of street-to-shop problem — identifying a fashion item in a user image and finding it in an online shop. Have you ever seen somebody in the street and thought “Wow, this is a nice dress, I wonder where I can buy it?” I haven’t. But for me, it was the cool task to try distance metric learning techniques. I hope that you will find it interesting too.

Dataset

Firstly, we need a dataset for it. Actually, I came to this idea after I found out that there are tons of images taken by users on Aliexpress. And I thought “Wow, I can make a search by image using this data, just for fun of course”. I have decided to focus on women’s top clothing for simplicity.

Below is the list of the categories I used for scrapping:

Dresses

Blouses & Shirts

Hoodies & Sweatshirts

Sweaters

Jackets & Coats

I used requests and BeautifulSoup for scrapping. Seller images can be obtained from the main page of the item, but for user’s images, we need to go through feedback pages. There is a thing called “colors” on item’s page. Color can be just item of another color or even completely other items. So we will consider different colors as different items.

“Colors” from item’s page

You can find the code that I have used to get all information about one item (it scraps even more than we need for our task) by link https://github.com/movchan74/street_to_shop_experiments/blob/master/get_item_info.py.

All we need is to go through search pages by each category, take URLs of all items and use the function above to get the info about each item.

Finally, we will have two sets of images for each item: images from a seller ( field url for each element in item['colors'] ) and images from users (field imgs for each element in item['feedbacks'] ).

For each color, we have only one image from a seller, but it can be more than one image for each color from users (sometimes there are no images for color at all).

Great! We got data. However, the collected dataset is noisy: