Making the Most of Black Friday

With Black Friday comes the craziness in price fluctuation.

The truth is that big ecommerce retailers, like Amazon, never stop changing prices. Monitoring these price fluctuations whether for competitor pricing analysis, construct pricing models or just get the cheapest price on that new tablet you wanted, is often a full time job.

Luckily, import·io makes monitoring prices super easy. No matter if you’re a small startup, large e-commerce retailer or just a data scientist like me thirsty for live web data, we have the right tools to help you on your data journey!

Putting it into Practice

So, what do you need to start monitoring prices with Import·io?

Well, first you’ll need to create an extractor for the prices you want to monitor. It only takes between 5 and 10 min!

Next, you’ll want to automate the data extraction by writing a script to query the extractor every X min/hours and save the data. Another 5 – 10 min if you know what you’re doing.

The other day, inspired by all the Black Friday deals, I decided to monitor the prices of tablets on amazon.com for last Thursday and Friday. Here is a quick and simple analysis (and visualization) of the data:

A) Prices of the different tablets over time B) Normalized prices of the different tablets over time. Normalization made by dividing all values by the mean price of each tablet. C) Boxplot of prices per tablet, showing prices quartiles and their variability D) Same as C, but normalizing prices by the mean price of each tablet (as in B) for easy comparison.

Amazon change prices for all the tablets frequently (plot A and B), usually oscillating +/- 2%. That may not seem like much, but occasionally you can see more significant the changes. For example, Nexus 7 prices were more than 3% higher than the average price at 3 different points, while the iPad mini was reduced to almost 10% below the average price. Bear in mind, that this is only 2 days worth of data and therefore there is not nearly enough of it to draw any real conclusions. The point I am trying to make is that with a few simple steps you can set up the framework for some pretty powerful price comparisons.

The Tip of the Iceberg

There are, of course, many more things we could do with this data. We could monitor the prices of each product across multiple retailers at the same time (by simply using more than one extractor). Or we could expand the current extractor to gather more than just price data: anything from ratings, to reviews, to product data, to specs, etc. All this extra data would open the door for all kinds of analysis. The opportunities to add value and insights to your business are unlimited!

How the Sausage is Made

Since Christmas is the season of giving, I wanted everyone to be able to replicate my methods and start monitoring prices as soon as possible. Here are the details of what I’ve done:

The extractor: I used a very simple Import·io extractor to get the price and reviews from the four most popular tablets, which was pulled live from Amazon’s website.

The script: I used a pretty simple Python script to automate the data extraction which you can see here. During my experiment, I set the script to run every 15 minutes on my local machine as a cron job – you can quite easily set a crontab in Linux/OSX or task scheduler in windows to run these sorts of tasks periodically. If you’re not sure how to do this, you can find a wealth of tutorials on the internet.

The data: I exported the raw data into a Google Spreadsheet which you can see here.

The plots: To give you the beautiful visualizations displayed above, I used Python Seaborn library which you can access here.