There are many organizations that have to contend with the technology platforms, architecture, and infrastructure required to support really big or really fast data. That being said, when I am asked about the challenges around Big Data, I remind people that big is relative to the organization's capability to derive analytics, insights, intelligence, and decision making around data better than their competition's efforts and results. Transforming to a big data organization requires constant evaluation of need versus opportunity versus capability.

So while really big data often starts with solving a number of technology issues, I suspect most organizations have a more significant challenge. Specifically, it is the challenge of harnessing the organizational intelligence around small data and dark data. I'm referring to data silos, analytics performed by Excel wizards (or spreadsheet jockeys), and other messy data practices.

Symptoms of Poor Data Governance

Do you email spreadsheets between coworkers to edit and review? Are there only a select few people in the organization capable of pulling or interpreting data out of core systems because of different data quality issues? Do presentations provide insights backed by data sources and assumptions? Does it seem like you have hundreds of reports and dozens of dashboards but none of them suit your day to day needs? Do you start a new analysis by cutting a new data set, or are you able to leverage defined tools to connect to predefined data repositories?

If you answered yes or often to many of these questions, then you probably have a need to consider improvements in your data practices.

Defining the Journey

The solution to these issues starts with better collaboration and partnership between the CIO, data driven business leaders, and internal data scientists. When one assembles this team, considers the data challenges and opportunities, assesses the culture, and recognizes technical and skill capabilities, I suspect that no two organizations are fully alike in developing their journeys.

There are principles that I can suggest. I'm a strong proponent of governed self-service business intelligence programs that aim to decentralize data scientists and outfit them with both tools and data practices. One can engage the organization and improve the culture around data practices by encouraging employees and managers to ask smart big data questions. Finally, the organization needs some driving principals on what data areas to focus on, but a good start is a CIO/CMO partnership especially concerning marketing data platforms.

Need a place to start? Like all transformations start with the low hanging fruit by working with the most collaborative departments around their opportunities with the quickest impacts on revenue or costs.