The more I read articles that purport to tell me why enterprises fail at DevOps, the more I realize that the conventional wisdom is off base. It's not that what the experts highlight is totally wrong, or that the issues they raise aren't important elements of DevOps in the enterprise. But these "Here are a few simple things holding you back, but once you do this you'll reach DevOps Nirvana" articles are fundamentally misguided and counterproductive, and they actively prevent DevOps progress. There's a better way.

So what's the real reason that enterprises fail at DevOps? DevOps is more than a set of tools. It's not a new set of practices. It's not all about automation or collaboration, continuous integration, continuous delivery, or X-as-a-service. And, despite what you read, DevOps is not fundamentally a new cultural paradigm. These things are all side effects and enablers of DevOps.

Automation, collaboration, and the reorganization of your culture are all elements of DevOps, but those are just tools for achieving DevOps' true purpose. DevOps is about producing value, and doing so with as little waste as possible. And to achieve that, you must eliminate anything that gets in the way.

To understand DevOps, start with Deming

The godfather of DevOps, Edward Deming, tried to get US industry to take up his theory of manufacturing. It didn't. Even after Japanese companies proved that his methodology worked, US companies weren't interested. Only after American manufacturers found themselves at a sustained competitive disadvantage did they want anything to do with Deming's Total Quality Management (TQM) system.

When the other guy can produce higher output and higher quality at lower cost, the outcome is inevitable. There was no better motivator than impending bankruptcy to get executives who had spurned Deming to suddenly develop an intense interest in his methods.

This is similar to the situation that legacy IT organizations, and the businesses they serve, find themselves in today.

The big fish no longer eat the small fish; the fast fish eat the slow ones, and this imperils every business beset with the inertia of legacy environments. "Adapt or die" is suddenly an imperative, and DevOps is a revolutionary adaptation to that maxim.

Revolutionary adaptations are both inevitable in their eventual dominance and in the resistance that they encounter. You see this in the fact that, despite its interest in TQM, the American automobile industry in Deming's time failed to succeed at implementing its principles for decades. Each round of failed experimentation became more costly than the last, and results remained elusive.

An entire industry was in danger of going extinct. And in the midst of this malaise, a call arose: "If Japan can, why can't we?" Certainly Deming had a few things to say on the subject. Among them:

"The Americans expect miracles. They want to copy Japan, but they don't know what to copy. They copy examples and then wonder what is the trouble."

and

"To copy is to invite disaster."

But Deming's views are best summed up in the following quote:

“Experience teaches nothing. In fact there is no experience to record without theory. … Without theory there is no learning. … And that is their downfall. People copy examples and then they wonder what is the trouble. They look at examples and without theory they learn nothing.”

We all prefer an easy answer to a hard one, and these corporations were no exception. Embracing the idea that conventional methods or "the way we have always done it" was wrong was too hard, and too scary. As long as there was any hope that those businesses could achieve results without taking that difficult road, they clung to it. They needed to believe that they could take the process examples from Japanese successes and fit them into what they already understood. This is what the majority of attempts at DevOps implementation in IT are doing today, and it takes a similar number of cycles of failure to learn that we can't jam square pegs into round holes.

Beware of easy answers

But there's hope. If you understand this you can pre-empt the costly cycle of failure that comes from an unwillingness to fundamentally rethink what you do and why you do it. It isn't inevitable that we must all fall into the comforting trap of settling for warm, easy answers when the unaccepted reality seems cold and harsh. The first one to unlearn and relearn gains the same sustained competitive advantage Deming's insights granted after World War II.

Meanwhile, there is a big market in telling people what they want to hear. Beware easy answers that prevent you from accepting the harder truth. Those answers might make you think, "If we start using these tools, if we adopt these practices, if we just change this couple of things, we'll be set for success with DevOps." This is wrong and only delays your progress.

What you need is to build a new foundation that supports new structures and alignments through people, process, and tools that are more agile and responsive to the markets in which you compete, and that's hard.

To predict DevOps success, look to your past

The best way to predict the future success of DevOps in your organization is to look to the past. DevOps is about nothing more than applying Deming's 14 points to all aspects of IT management, and treating your software lifecycle like a manufacturing process.

Deming's ninth point, "Break down barriers between departments," is a pretty good summation of what the term "DevOps" conveys. It was the original basis for DevOps.

Incorporating the 14 points will lead naturally to the creation of the new tools and techniques so commonly discussed in many DevOps articles. But you can't just skip a step and apply those tools and techniques to your environment and then get the desired result without first embracing the principles of the transformative movement that created DevOps. As Deming said, "Managing for result is like driving by looking in the rearview mirror. ... You have to manage the cause, not the result."

There are good reasons your organization does things the way it does, although you may have no idea what those reasons are. How can you manage a cause you haven't uncovered?

You can't. That's why the vast majority of articles written on enterprise DevOps are insufficient. The advice given brings to mind the directions an elderly gentleman once gave me long ago, when I was lost in the hills of northern Alabama:

You can't get there from here. You have to start somewhere else.

Deming was a statistician who understood variations in quality. By crunching numbers he made his most important discovery: "The greatest waste is the waste of human potential." So he developed his System of Profound Knowledge, which has four lenses: systems thinking, understanding variation, epistemology (theory of knowledge), and human psychology. From analyzing process according to these lenses, the 14 points arise, and the lens Deming pointed to as most dominant is also the most neglected today. It's the fourth lens: psychology.

To understand why we grapple with throwing out the old and embracing the new paradigms of DevOps, you must first understand why you do things the way you do now, and why it is so hard to challenge that received wisdom. We have all accepted that construct, but we seldom consciously acknowledge it, even to ourselves. We absorb these cultural constructs so that, like the ego floating on top of the id, they become unconscious drivers of our behavior that we do not understand. They result in various organizational pathologies until we bring them up to the light of day and make a conscious decision about who we want to be.

It is both important and good that culture, "the way we do things around here," and "the things we don't do or even talk about" are powerful and settle into the unconscious nooks and crannies of our minds. Both societies and organizations crumble due to cultural dissonance when alternative ideas about important constructs emerge, and change can take generations to take hold.

Most of the ideas that shape our current business cultures in the West are inherited from a management methodology called “Taylorism,” or “Scientific Management.” The precepts of this methodology are in direct contradiction to nearly all of the 14 points. DevOps and Deming’s points are not additions to the Taylorism by which we run things. They’re replacements.

We are so steeped in this mindset and culture that we never question whether or not this is the only possible way, never mind if it is the best way. The answers are, respectively, no, it isn't, and no, it isn't. This management methodology inherently and necessarily violates nearly all of Deming's 14 points, which apply to DevOps as well as they do to TQM. This is the real problem with which you are grappling. That is the round hole into which you are trying to pound the square peg of DevOps.

Start with the 14 principles

If you really want to succeed with DevOps, there are two places to start: Take a deep dive into the underlying principles of DevOps and why they work, and then take a deep dive into your own organizational culture and its unique expression of itself as process. Figure out why you do things the way you do. When you look with the right lens, you'll be surprised at what you find.

Keep learning