Big Data is showing the first signs of heading for Gartner’s Trough of Disillusionment as a spate of voices, some very well known, like Nassim Taleb, author of Antifragile and other prescient books, begin to question the hype. I’m going to show you why Big Data is in the Trough in some ways and beyond that, and on the Slope of Enlightenment in others.

The Hype Cycle

In case you’re not aware, Gartner publishes a hype cycle for emerging technologies meant to give us an idea of when we should expect new technology to be commercially viable. With all of the noise in the marketplace fed by both vendors and media, it would be easy to think anything less than the latest product offering is already antiquated and won’t work. Gartner’s Hype Cycle helps to bust that myth.

Because the truth is much different. Most companies can’t afford to chase their technology tail and most IT departments can’t suffer the lost credibility of deploying technology that isn’t ready for prime time. All technology goes through a hype cycle that follows the pattern in this image, even if the peak may not be as high or the trough as deep…or it could be worse.

Big Data’s tracking along the Hype Cycle

Getting back to Big Data, we’re seeing a technology pattern that is certainly evolving. Gartner has five ‘key phases’ that determine where technology falls, and it would be worthwhile to break down Big Data’s circumstances using Gartner’s familiar language:

Technology Trigger: The technology breakthrough that brought Big Data to the technology stage was the development of Hadoop. While it is a usable technology, it wasn’t developed for public consumption and is challenging for organizations that don’t have strong IT support. Hadoop came about to satisfy the demands of web scale problems like search and social media.

Peak of Inflated Expectations: There were certainly early success stories from the companies that developed and first used Hadoop like Google, Facebook and Yahoo. Their success launched a series of Hadoop-based solutions that attracted VC investment, including companies like Cloudera, Datameer, Platfora and SiSense. Likewise, there has been fast growth for powerful analytics solutions that allow business users to manipulate and visualize enormous data sets. The marketplace doesn’t lack for solutions.

Trough of Disillusionment: In some industries and use cases, we’re at the point where the rubber meets the road…the point where Big Data delivers or doesn’t. What happens now determines if we’re extending the peak of expectations or starting the slide into the Trough. Patience with experimentation will wear thin over the next year or so and there need to be more ‘everyday’ companies taking advantage of Big Data and talking about their successes. Big Data is headed for the Trough as long as there are more people trying than succeeding, and that’s where we are right now.

Companies can only sell the idea of “uncovering hidden insights” for so long. At some point, Big Data needs to be operational and automated.

Slope of Enlightenment: Curiously, in some ways Big Data is on the Slope already in some areas. There are use cases around pharmaceutical clinical trials, customer preference, transport and logistics, healthcare and others. In some cases, second and third generation products are here, especially in the analytics space. There are many home-built solutions. Where Hadoop-y vendors need to be careful is simple: Big Data was here before the hype. People were solving the problem without Hadoop, believe it or not.

You might not realize from the hype that many of the organizations with demonstrated success on the books were managing Big Data before Hadoop was ever developed. For them, Hadoop isn’t a breakthrough and is just a choice among many. This is where you’ll see Big Data hype geting the most eye rolls and groans.

The irony of Big Data is that the places where success is quite clear are often the places where the term simply isn’t used. In those places, the term may actually be disliked and kept at arm’s length.

Plateau of Productivity: This is a far-off land at the moment all but those just mentioned. The mainstream technology buyer is struggling with the plethora of messages and products, suspicious of the arguments that make Big Data sound too good to be true.

Nearly on year ago, Gartner had Big Data nearing the Peak of Inflated Expectations (see below). It will be interesting to see where they place Big Data for the report due out this Summer.

Chris Taylor is a marketing executive with TIBCO, and cofounder of Successful Workplace.