Meet John Edwards, the owner of a growing online business in Chicago. He is into selling local, vintage and customized wooden furniture on sculptwoodz.com. After a steady start, business really starts picking up. In fact its doing so good, John plans to diversify on a large scale. With a 500-people strong team, right from online sales management personnel, skilled wooden furniture makers, sales & service helpline personnel, packaging and delivery personnel, he is all set to go bigger. It is only when expansion plan details are further studied, that things look complicated. What else would people procuring elegant wooden furniture like to buy? Customized pillow covers and cushions? How about coffee mugs? How about carpets? Moreover, sculptwoodz.com is looking to expand sale in regions around the vicinity - Cincinnati, Cleveland, Detroit and Philadelphia over a reasonably short period of one-two years. This involved data gathering on a large scale.

A close observer of technology and market trends, John realizes that Big Data analysis would be the apt answer to his problems.

The following issues assail him:

1. There are several customer-demand based questions with regard to business expansion, that need immediate answers.

2. He has no idea of the market base, or what the customers are like with regard to made-to-order home and office interior products in Cincinnati, Cleveland, Detroit and Philadelphia.

3. Also, this mammoth amount of information is required fast. The business can't afford to wait for six months or more with other online competitors looking to surge forward.

4. It is finally decided that a detailed, clear Big Data analytical report would be required within couple of months or less. This will help John formulate a definite action plan and beat the competition.

Hiring Big Data Experts

"By 2018, the United States alone could face a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions." - A McKinsey Global Institute Study.

Much to his surprise, John finds that a Big Data Solutions expert is a hard person to recruit. The required skills ensure that it is a niche job opening. Demand is high and supply, low. In fact, many companies are still figuring out what people recruited to the two most in-demand data jobs in 2015 - Data Analyst and Data Scientist must be handling. Anyway, John looks for a Big Data team with the following qualities:

1. Cleaning, importing, validating data for arriving at conclusions; making business decisions on its basis.

2. Mathematical expertise and business instincts – data has layers, dimensions, textures and patterns that require such a skill-set.

3. Deriving conclusions from numbers, making analytic models using various mathematical techniques.

4. Presenting researched data for easy understanding in the form of charts, tables, graphs, etc.

5. Ample technical skills to solve and build clever solutions to problems. An algorithmic bent of mind to ensure purposeful data rearrangement.

6. Expert knowledge of Big Data analysis tools including popular and prevalent ones like Hadoop, MapReduce, R, SQL and SAS, apart from other advanced data-focused skills.

7. A business mindset would work wonders. For, Big Data results is not just numbers, but of applying strategy, based on the numbers. Using data to effectively narrate a story, a clear problem-solution interplay, data insights for support, were other desired qualities.

Data Experts at Work

"If you do not know how to ask the right question, you discover nothing." - W.Edwards Deming, American statistician, author and management consultant.

Finally, post carefully held interviews with a prime Big Data expert as the interviewer, an initial team of five Big Data experts are hired. These 'Data Experts' as they are called, begin by gathering source data for the purpose. But as in any effective Big Data project, it goes down to asking the right questions, on which the analysis would be based. To list a few:

1. What percentage of the residential and office population were into buying customized wooden furniture online?

2. What other customized products did this select section of customers buy online apart from furniture?

3. What were their specific preferences, with regards to colour, size, utility, shape and price of the purchased items?

This is how the Big Data team went about it:

1. Based on the queries listed post their interaction with John and other key employees, a huge data gathering exercise is undertaken. In this case, online customer behavior of similar product buyers in Cincinnati, Cleveland, Detroit and Philadelphia prove to be a prime data source. Presently, US organizations accumulate data from varied channels. (Email - 63%, Website – 62%, Point of Sale – 47%, Call Centres – 47%, Social Media – 29%, Mobile Application & Devices – 27%. According to estimates, in the upcoming five years, the world will be searching Websites (65%), Mobile Devices & Applications (62%), Social Media (60%) and Email (55%) ) for data.

2. Post data gathering on a large scale, the team moves on to Big Data Market Research. Behavioral research and advanced analytics combine to offer faster and high quality results. Finding the key reasons for observed behavior (What is making the customer behave in a certain way and why?) is the key to decoding potential online customer habits. Suitably chosen data tools play a major role in the process.

3. Studying online customer transactions (what was purchased, when, at what time, etc) helps crack the code with regard to precise customer actions and reactions to the user interface, product display page and the UI.

4. The last but most important step is clear representation of analytics with the help of visuals, graphs and single-line statements. This way, John doesn't have to dwell over intricacies. He is served his exact requirements in a concise document.

5. Acting on the Big Data analytics report, John Edwards was able to push sculptwoodz.com to new heights, thanks to timely Big Data intervention. Online surveys to find customer preferences, keeping a constant record of purchase transactions, were among the many steps taken to know the customer better.

6. Cut to present day, even as John's online business continues to expand, the Big Data team manages data-related assets of the entire organization. Standards have now been set for data monitoring. A clear cut approach to data profiling, monitoring and visualizing is another striking feature of sculptwoodz.com's long-term Big Data analysis team.

Stats Corner: Big Data Statistics for 2016 (Highlights)

1. Inaccurate, outdated, incomplete, inconsistent data are the modern-day hurdles of the big data world. 75% companies believe inaccurate data will bar them from providing a great customer experience. About 23% of all collected customer data is inaccurate. As far as data errors by businesses go, more than 50% is attributed to missing/incomplete data, duplication of data, inconsistent data and outdated information.

2. 'To completely know the customer' can sound creepy, an overwhelming 97% of US organisations are looking for the same. Their main intention – increase customer retention and sales, accordingly make better decisions, provide better (customized) customer experience, cut down on costs and finally understand the legalities of customer base data and detailed analytics of the same.

Last Word

How would you leverage the potential of Big Data analytics for your business? What problems did you face while implementing Big Data findings? You are welcome to share your experiences as comments to this article. In the next edition of this article series, we shall talk about Big Data and its role in the insurance sector.

(Big Data statistics featured here sourced from The Experian Data Quality Nov 2015 survey.)