Big data has been around for decades but has only taken on the form of a buzzword in the last few years. So why are people taking an increased interest in it? One word: money.

Big data is nothing new. In fact, although the official definition – which refers to big data in terms of data volume, velocity and variety – only came about in 2001, companies have been gathering large amounts of data for decades. But big data has taken on a new lease of life in the last five years, largely as a result of companies finding new ways to analyse data.

Experts at GP Bullhound, an investment banking firm, suggest the future of big data in a new report, entitled 'Extracting Insights from Exabytes', released today:

We believe the analytics market is entering a new era, where technology is capable of supporting data-driven business, in real-time.

Really, big data has moved on from the initial stage – where the challenge was about storing the data – and has moved onto the next, which is all about the insights companies can obtain from the data.

So just why are more people than ever searching for big data? Here's five reasons:

1. Unstructured data has never been so ubiquitous

Facebook Twitter Pinterest A minute in the life of the internet. Source: GP Bullhound

One of the elements that makes big data, well, big, is the data type. Unlike traditional business insight, which analyses structured data (the likes of which include financial details, sales and inventory), big data analytics tends to focus on unstructured data, such as emails, videos, photos and even posts on social media networks.

According to a 2011 IBM report, IBM Big Data Success Stories, 90% of the world's data was created in the two years before publication. Here are some figures to help you understand just where such data is coming from: every minute, 208,300 photos are uploaded to Facebook and 350,000 updates sent on Twitter. Businesses can use this data to understand whether customers are speaking about their companies in a positive or negative way, for example.

2. Tools such as 'Hadoop' mean that storing large amounts of data has become incredibly cheap

Although unstructured data is becoming more pervasive Hadoop, an open-source framework for storing large scale data, has developed substantially in the last decade. No longer a research project, Hadoop underpins data processing at some of the world's largest internet businesses. Why? It can deal with unstructured data and it is faster and cheaper than tools before it.

Take the figures in the chart: Hadoop processes a terabyte of data at $333 (£205), far cheaper than any of its rivals.

3. Now it's cheaper to process the data, companies are getting real insight from it

Facebook Twitter Pinterest US retail outfit Target knew a woman was pregnant before her father was aware. Photograph: Amy Sancetta/AP

Retail stores are one of the best examples of companies that use big data. What are they analysing? Customer activity using loyalty cards. Supermarkets then use this data to influence in-store decisions. For example, they might stock a low-selling cereal because the customers who tend to spend the most have a tendency to buy it. Without this knowledge, they may have lost out on potential revenue or, worse, lost customers.

But loyalty cards – and even debit cards – give companies very detailed insights into individual customers. This proved to be particularly true when last year a father turned up to a Target store furious that his daughter had been sent coupons for baby clothes.

The New York Times reported the conversation:

My daughter got this in the mail!” he said. “She’s still in high school, and you’re sending her coupons for baby clothes and cribs? Are you trying to encourage her to get pregnant?"

A few days later, he apologised to the store manager after finding out about the pregnancy.

It later emerged that Target had been analysing shoppers' behaviour to analyse whether a customer was expecting a baby. If they started to buy supplements such as calcium, magnesium and zinc, the store would assume that they were getting close to their delivery date.

Anyone who has ever bought something on Amazon will have noticed this, too, in the form of "customers who bought this item also bought...". What kind of impact does that have on sales? 35% of Amazon’s sales come from product recommendations, according to GP Bullhound's report. Similarly, Walmart used big data analytics to increase completed online sales by approximately 10-15%.

4. Big data analytics could lead to $610 in productivity gains in only four sectors

If there was mainstream adoption of big data analytics, the retail and manufacturing industries alone could see an increase of $325 billion to their annual GDP as a result of increased efficiency, according to a report by McKinsey in July. Healthcare and government services could also see productivity gains of as much as $285 billion by 2020.

5. Big data analytics saw nearly $1.4 billion of VC funding in the last 12 months

Data analytics fundraising by quarter Photograph: GP Bullhound

Venture capitalists have started to look at big data analytics with increasing scrutiny. In the last 12 months, they invested $1.37 billion into various companies, an increase of 217% over investment in the previous period. There were 19 deals in the last quarter alone, according to GP Bullhound's report.

There are a few reasons to suggest why this increased investment is taking place:

Big data is now "enterprise-ready", i.e. it is commercially useful. It is now cheaper to store and process this data and increases in computer processing speeds mean that more businesses can leverage big data analytics. Analytics tools are opening up big data to people without specialised skills. Although only PhD-level specialists could understand the earliest versions of tools like Hadoop , new iterations and companies are democratising big data. The most-common feature is for companies to show the data in easy-to-understand visualisations. Analysis can now be done in real-time. While Hadoop was not designed for real-time analysis, new companies are now innovating to build on the framework to give companies instant insight. Such technology is being used by companies like Hailo, the taxi-calling app, to assign drivers to prospective passengers.

Why do you think investment in big data analytics is increasing? Continue the conversation on Twitter with @sirajdatoo and @guardiandata.