One data a time is big data at culmination – A thorough analysis why big data requires small data

Unless you had been living inside the caves since past few years, you must have heard how big data – the shiny new catchphrase of the marketing world – has literally taken over the world. Bit by bit, step over step, day after day; big data is only growing bigger and it is expected to streamline every single spectrum of the marketing world. It aims to regularize order, bring simplicity and improve accountability and transparency in the way the world works – with the most lethal power ever invented – data

Big data have had tremendous press (both online and offline) coverage – so much so that – the information surplus often creates confusions and misinterpretation. One quote that best summarizes this point – even if you have read it elsewhere earlier – is this:

Among many confusions or should I say misinterpretations, one apparent theory is big data is big data and it is vast, huge, uncontrolled and magnificent. While part of the statement is a fact- there are many things about it that many of us need to rethink, re understand and re conceptualize.

First, if we say big data is all about big data; we are dragging it into the hole of identity crisis.

Data scientists – the cool gentle folks with six-figure income of the century – dive into the data ocean of pinpointed information and distill it into highly specific and actionable insights ready for immediate consumption for marketing purpose. The data serves internal customers, refining, filtering, clustering and refreshing it every time it is processed to ensure it always is relevant.

However, when they process the pool of endless information, they are actually dealing with single subset of meaningful small data that collectively constitute the broader picture – big data

Why big data is about small data/ why big data needs small data

So, small data when clubbed together becomes big data. In order to get into the basic of it’s essence, let us cluster the topic of analysis:

1 . Small data and information

2. Why small data is important with example

3. Challenge of expressing emotion

4. Bragging of information

5. Information and insight

Small data is data in a specific set of volume and in regulated format that is informative and actionable; allows the stakeholder of it to easily access it and then use it step by step.

If small data does not pinpoint basic information, big data set is non-existent

Small data is a specific set of datasets; which is populated with extremely specific attributes. It is used and analyzed to measure the present state and conditions or may be generated by pinpointing larger datasets.

Small data is important because it can trigger events based on what is happening now. The events can then be merged with behavioral information derived from specifically clustered sources such as machine learning algorithms. So, big data is almost non-existent without the filler offered and provided by small data.

Think about a wind turbine as an example of small data – how it compliments big data patterns

It has a range of sensors strategically mounted on it to analyze and assess many activities including speed, wind movement, velocity, temperature, vibration etc; to name a few. The turbine’s blade is so programmed that based on the feedback provided by small data, it can adjust to changing wind condition. These small data sets are interlinked with large data lakes where machine learning start to understand pattern from the information gathered from small data sets in order to accomplish a range of duties; namely predicting the shelf life of a parts, helping a product with basic maintenance for strategic functioning etc; to name a few.

It is very challenging to express emotions using big data compared to small data

Let us take an example – Small data is more powerful than big data. We will give an example to prove it. More than a decade ago, Lego Company believed instant gratification generation would kill their product and hence they changed the size of their product – small, tiny bricks – to huge building blocks. Very soon, the company was almost going into a bankruptcy mode. Later; they changed the size of the bricks back to tiny ones, invented the Lego Movie and today is number one.

The problem with organization is they can’t take decision in the absence of near perfect data; they brag they have lot of data – which if not used and only talked about – actually a dumb data.

Today organizations brag they have data – ocean of data – which can be leveraged on to get meaningful set of specialized and insightful information. But do they actually use it?

Huge set of data means lack of near perfect data – either because there is the lack of skilled resources to extract value from it or skilled people do not necessarily know how to extract useful information in a specific environment from that data set. In this case, big data turns out to be dub data.

That is exactly where they need to value small subset of pinpointed information in order to use them for organizational value. In other words, that’s why big data needs small data!

Big data provides information. Small data provides insight

Big data as we mentioned is made up of small data. Small data is the bit sized; evenly balanced; and strategic; ready to become useful information – offers insight – pinpointed statistics and basic patterns which when collectively processed into greater and streamlined information system with the help of tools and technologies becomes big data.

So, emotion and context which generate value and meaning can be sourced from small data rather than accumulated information reflected through big data.

To make total sense in terms of applicability and usefulness; big data therefore needs the adequate helping of small data. Without small data, big data is non-existent; One data a time is big data at culmination!