Massive disruption is occurring as marketing goes digital. Business is moving steadily towards providing a fully personalized and truly integrated digital experience—building upon recent advances in user experience, analytics, cloud computing and storage, and an omni-channel experience across mobile platforms and social media.

Yet while the end user digital journey has improved significantly, we are only in the earliest stages of offering a compelling, integrated digital experience. The biggest impediment? The accelerating rate of change around mobile, web, and social experiences. Existing technologies struggle to keep pace, while new technologies fail to offer either the breadth or depth of the capabilities required. Traditional paradigms in software development are failing to keep up.

The degree of personalization needed for the next generation digital experience requires managing a lot of data. Superdata. As the degree of personalization and anticipation of the online visitor’s needs increases, computing power also must increase. This is where open source is taking the lead. The vision for this next generation digital experience has been emerging over the past few years. Forrester started out calling it Web Experience Management.

More recently, as "web" began to sound a bit less cutting edge, they’ve begun to call it Digital Customer Experience, or Digital Experience (DX). Others call it Digital Experience Management, or DXM.

The good news is that many of the technologies that will allow us to accomplish this goal have arrived on the scene and are now being put to the task. What is also clear is that open source is the foundation on which this next generation digital experience is being built. Big Data is the term given to the idea of gathering, storing, and mining all of the information required to achieve a desired result. Personalization is the process of applying this information to the individual.

Companies like Jaspersoft have made Open BI, or business intelligence, a reality in the quest for analyzing and interpreting the megadata required. 10Gen, the company behind MongoDB, a NoSQL database, has the kinds of tools that are needed for managing all of that data. And underneath it all, Nagios is required for monitoring the rapidly expanding IT infrastructure needed to accommodate large-scale processing. These solutions are all built on open source.

And today, we cannot think about Big Data without thinking of Apache Hadoop. Named after the stuffed animal of the creator’s daughter, the Hadoop framework allows for the distributed processing of large data sets across clusters of computers. Hadoop is designed to scale up from single servers to thousands of machines, providing the computing power to handle data in the enormous volumes generated. There are new Hadoop-based growth companies like Cloudera, Hortonworks, and MapR.

Established proprietary vendors have even entered the open source Hadoop world, including Oracle with Big Data Appliance; EMC with Greenplum; and IBM with InfoSphere BigInsights. Even the historic naysayer of open source, Microsoft, has embraced open source with Big Data Solution.

Massive amounts of data are crunched to understand both individual experiences and the broader community to which they belong. Historical data can be used to extrapolate the best path and optimal choices for a specific individual. Context is yet another key to making the visitors feel at home. Why is the visitor on the site? Are they shopping for themselves? Or shopping for a relative? And what choices have they made that might influence the next step in the journey? Will the site react with a logical next step that is influenced by what has come directly before, or will the next step hit a false note?

Because this next gen DX is all about innovation, open source has become essential to its progress. Open source platforms such as Drupal are technology’s reaction to digital disruption and have enabled agility and rapid innovation in a disrupted climate. Open source’s "open" philosophy is one in which technical goals and challenges are shared publicly—and globally—rather than being addressed by a small, discrete workforce at a single company or organization. The global audience of collaborators, eager to take up the challenge, drives this essential progress at an astounding rate.

Not surprisingly, unlimited scalability, as well as the freedom that comes with open data and open architectures, is essential in this pioneering territory. The vast, expandable resources and storage capabilities of the open cloud are an essential enabler in building innovative, agile, and scalable experiences. Being able to build, scale, and fail easily, and move to the next stage with ease and speed, means that there is no obstruction as new territory is explored.

And because we have the power of open source at our backs, we can envision the next generation digital experience as being truly integrated. It must be of a piece across all platforms: laptops, tablets, smartphones, iWatches, and whatever comes next.

No longer do we need to artificially separate digital experiences into social, content, and commerce contexts that are awkwardly cobbled together. Instead, a single experience must offer all aspects of interaction. Open source enables this as well.

As we watch the next generation digital experience develop, we will continue to see open source at center stage of this digital revolution. Without the collaborative freedom of open source, the next gen DX would simply be a vision, not a rapidly emerging reality. And Hadoop would only be the name of a child’s stuffed animal.