The ongoing industry focus on digital transformation remains the hottest and most strategic conversation in the business of IT today. You can't read an article or go to a conference without getting an earful on the subject these days, and for good reason: The world has become far more digital, customer tastes and expectations have changed dramatically in recent years, and so much more is technically possible today than even just a half-decade ago.

Businesses have finally gotten this message. Most data now show that plans are underway to upgrade their customer experiences, rethink their products and services, and consider new markets and business models in the majority of organizations, even if it's still early days in the overall journey.

However, in talking with many larger enterprises over the last few years, I've noticed -- and the available data further confirms it -- that the digital art-of-the-possible is very much pulling away from average practice. There appear to be two primary causes for this: a) Lack of sufficiently strategic and innovative digital change/realization at scale and b) a firmly entrenched set of institutional barriers to doing something very new and different than what the organization did before.

Overcoming the Obstacles to Digital Opportunity

The first category of barriers is being able to think and act effectively as an organization in the digital marketplace, well outside of the traditional box, regardless of what's holding you back. And the second category is a laundry list of barriers that have become more and more apparent in the industry as organizations attempt major realignments and reinventions around digital concepts and technologies to a greater degree than any other time in business history.

There is good news to report: Overall, I believe we are making solid progress today on the first category of barriers. The industry conversation around digital change agents and more decentralized transformation is a good indication of this. We've learned to seek out those with good ideas, then enable and empower them to try and experiment, and to do this at scale. That's because only organizations with sufficient experimental capacity are going to survive, by pathfinding the way through the still-nascent and rapidly evolving digital jungle.

This progress is being combined with advent of a new generation of IT leader, one that is much more concerned with the impact that IT has from an experiential and business standpoint. I recently profiled what this new CIO looks like and it's a model that is more focused on stakeholder enablement, satisfaction, and seizing digital opportunity, even though topics like cybersecurity and data privacy still remain very high priorities.

However, clouds have appeared on the horizon, and not just the proverbial digital ones.

Particularly problematic appears to be the realization that at least in some organizations there have grown nearly-intractable barriers to real change. Some of these barriers are well known, such as the cultural and skill impediments to becoming more digital. Fortunately, as we're beginning to learn, we're developing ways the digital skills shortfall can be addressed. But corporate culture remains one of the most stubborn obstacles -- even as it can also be one of the greatest enablers -- to the kind of cross-functional change and restructuring that must take place in most organizations.

Legacy Decisions Loom Over Digital Transformation

Other barriers to digital transformation are starting to look a bit more serious for some of us. One is the fact that even though all companies are becoming more like technology firms, they have not nearly been matching the levels of investment that technology companies actually make (see the first graph, here.) This means they've been underinvesting by up to 50% in IT year-over-year and are potentially far behind their digital native counterparts. That's a lot of catching up to do that's not possible to compress into a single sustained effort in digital transformation, even if very well funded and resourced.

The second barrier is very well known in IT, software development, and enterprise architecture circles, but almost entirely unknown outside of them. It's the concept of technical debt, which in real terms has been steadily accumulating in organizations for about as long as they've had their existing platforms in place. The short version is that technical debt is incurred when the more expedient option is taken during the technology development process over the smarter, better choice for the longer term, but which often takes more time and resources to realize.

How can so much technical debt accumulate? Unlike physically constructed objects, the code, design decisions, and architectural choices in digital systems are far more intangible and harder to discern until shifting operating conditions make them readily apparent, often suddenly and unexpectedly.

To explore this issue, I've been having numerous industry conversations lately about the growing amounts of technical debt across the IT industry, much of which -- how much exactly is the central debate -- has to be paid down first if digital transformation is to take place.

Few Shortcuts to the Digital Future

That's because even though some technical debt can indeed be ripped out and replaced wholesale during the transformation process, much of it -- such as fragmented customer databases, systems that haven't been upgraded in recent memory, time-consuming operational manual processes that should long ago been automated, etc. -- cannot quickly or easily be fixed. Yet very quickly done it must be according to my survey of CIOs earlier this year.

A few choice quotes from my conversations this week with several leading IT experts will serve to illuminate the scope of the technical debt issue. This first is from Wayne Sadin, COO/CTO at Affinitas Life and a well-known expert on CIO issues:

The serious and underappreciated topic of technical debt is a board-level risk for many reasons. Technical debt is an 'off-balance-sheet liability' that potentially dwarfs Enron and WorldCom in size. Boards have no clue how bad it is.

The second is from Tim Crawford, an industry colleague and CIO maven:

You can only build up so much technical debt until it collapses under its own weight.

The last quote is from an actual CIO of an educational institution, Kyle Johnson. It is perhaps the most sobering, as even though he was being tongue-in-cheek on the subject -- and kindly agreed to let me reproduce this quote in that light -- it shows how the conversation is a non-starter with those outside of IT, and often even harder for them to understand:

I'm informing our leadership about our significant, crippling technical debt. I'll let you know tomorrow if I'm looking for another job. '-)

These quotes are fairly representative of the industry conversation in my experience. As a result, a growing number of IT leaders are starting to suggest that organizations more rigorously measure and manage technical debt. Yet the reality is the the vast majority of enterprises do not do so today, and so have essentially no handle on the size of the hole they're standing in as they embark on their digital transformation journey. Getting a clear handle on technical debt is an objective more boards of directors need to push for more urgently as they sign off on rapid action plans for the next-generation of their business.

There are other barriers to digital transformation as well, including high degrees of bureaucracy and centralization, and overriding focus on controlling IT costs, instead of spending on opportunities and becoming more of a P&L, and a poor integration foundation -- especially without a data-based microservices architecture -- upon which to build the next generation of customer, supplier, and workforce experiences (which can take years to get right first.)

However, these barriers have clearer solutions and are probably significant smaller in size that technical debt (the integration foundation excepted.) In my estimation, underinvestment and technical debt remain two of the highest hurdles to moving forward successful with digital transformation.

How CIOs can overcome key barriers to digital transformation

While there is no silver bullet (there rarely is in the technology world), there are some clear steps that IT and business leaders can take to get a handle on and address these challenges:

Institute a process to begin measuring technical debt. Although a fraught topic with low maturity across the industry, not starting is a good way to never resolve the issue. It's a touchy topic, but necessary in most organizations to embark on meaningful digital transformation.

Although a fraught topic with low maturity across the industry, not starting is a good way to never resolve the issue. It's a touchy topic, but necessary in most organizations to embark on meaningful digital transformation. Have a frank discussion about overall IT spending using benchmarking as a discussion. Do you want to be the Amazon, Netflix, or Airbnb of your industry? You'll likely need to invest a lot more than you are today and it will have medium-term impacts to the bottom line, as you're competing with companies who understand this. It's a board level discussion. And don't forget to include line-of-business spending and Shadow IT in the conversation.

Do you want to be the Amazon, Netflix, or Airbnb of your industry? You'll likely need to invest a lot more than you are today and it will have medium-term impacts to the bottom line, as you're competing with companies who understand this. It's a board level discussion. And don't forget to include line-of-business spending and Shadow IT in the conversation. Experiment prolifically in digital and provide air cover to those that don't succeed. Considerable research now shows that the risk takers are the ones that win in digital. Enlist change agents, but very importantly, protect them if they are not successful at first. No one will come after them if failures are not looked at as key and necessary lessons in digital. Spread those lessons as they are learned across the organization.

Additional Reading

How digital transformation is reshaping the IT budget: The journey of 3 CIOs

How can businesses keep up with tech change today?

Closing the gap between executives and digital transformation