The title of this piece is heresy. The idea of self organizing teams is a center pillar of the Agile movement.

Or is it? The Agile Manifesto says,

The best architectures, requirements, and designs emerge from self-organizing teams.

This statement does not say, “Always use self-organizing teams”, or “Self-organizing teams always work best”. Yet, it has been interpreted that way—grossly misinterpreted in fact—by much of the Agile community.

What the Manifesto says above, if you read it literally, is that when you look at the best architectures, requirements, and designs, they will have come from self-organizing teams. That statement is an absolute, and so it cannot be correct, because absolutes about human behavior never are, but the intention is sound—that self-organization can be powerful.

But it does not say that it always works, or that it is always best.

It is also important to remember that while the Agile Manifesto is brilliant, and has largely stood the test of time, it is not perfect: it was not burned into stone by the Heavens. It was created by a group of people, and it can be wrong.

So what is the reality?

When Self-Organization Works, and When It Doesn’t

My wife is a behavioral therapist, and when I first told her that Agile advocates that teams self-organize, she looked at me like I was crazy. She deals with people’s problems, and they tell her what they really think about their family, friends, and co-workers—and she is all too familiar with the dysfunctions that people can have in interacting with others, and the many forms of cognitive bias that exist among humans.

The Tuckman model is often cited by the Agile community to describe “forming, storming, norming, performing” phases of a new team. However, the original Tuckman paper stated that 50% of the teams studied experienced conflict within the group, and that among those, the duration, intensity and destructiveness of the "storms" could be varied. In the words of Tuckman himself,

“The second point in the sequence is characterized by conflict and polarization around interpersonal issues…”

In other words, half the time, new teams experience serious conflict. It is important to consider how these conflicts might resolve themselves:

In a self-organizing team it is not uncommon for a leader to emerge, or perhaps more than one, and that those ad-hoc leaders achieve their de-facto leadership role by dominating discussions. Those who are quiet and thoughtful will often shut down when there are aggressive and vocal people present. In that situation, the more quiet members of the team will feel frustrated and disempowered, and the team will underperform. From this Smith College summary of the Tuckman model,

“These reflect conflicts over leadership, structure, power, and authority. There may be wide swings in members’ behavior based on emerging issues of competition and hostilities. Because of the discomfort generated during this stage, some members may remain completely silent while others attempt to dominate.”

That is, when a self-organizing team begins to “perform”, it could be through the psychological domination of some members. The team appears to be working, more or less, but there is hidden deep discontent.

Indeed, if one looks at human history, leaders who have emerged through self organization have often been the power-seeking, from Caesar to Genghis Khan and Hitler. It is mostly only when there is a framework of constraints for how leaders can be chosen and are allowed to act that we tend to see leaders who behave.

My Own Personal Experiences

Agile coaches often work in teams. As it turns out, just as “the cobbler’s children have no shoes”, I have found that a surprising number of Agile coaches do not practice what they preach in how they personally deal with others who they are teamed with. I have encountered many Agile coaches who are not collaborative, but instead are competitive to the point of not sharing information that would benefit the other coaches.

In a recent case, a coach who is a professional Scrum trainer had been put in the role of a product owner, and she refused to cooperate with me in any way. I am not sure why; perhaps since I am an Agile and DevOps coach, she was afraid that I would interfere. Regardless, it was so bad that I ended up leaving the engagement.

Another case was particularly awful. I was part of a three person team stood up by the CIO of a major government agency to assist him with an Agile and DevOps transformation. The plan was for there to be a government section chief to oversee us, but for the first six months that individual had not yet been hired, and so during that time we reported to the CIO (which enabled us to be very effective, by the way).

I later learned that one of our team had previously applied for the section chief job, and had been rejected for it. Yet he still wanted the job, and thought that he could impress the CIO and others so much that they would reconsider and give him the job.

I did not know this, but it explained his behavior, which—from the beginning—was to exclude me from his meetings, even though we were on the same team. He also would do his best to take over every meeting that we shared, so that I could not get a word in. He had an aggressive personality, with a booming voice, and so it was difficult for me to hold my own, even though I had far more experience and knowledge with respect to Agile, DevOps, and IT in general. It was a nightmare.

Overall, my own experience being on teams that had no designated leader has been mixed—I definitely would not characterize it as good. I am therefore skeptical of the notion of self-organization. Being somewhat introverted, a book that really resonated with me was Susan Cain’s Quiet: The Power of Introverts in a World that Can’t Stop Talking. That book explains why extroverts and aggressive people can effectively take over when a situation is unstructured—whether intentionally or not—silencing those who are less aggressive: it is like having a lion among lambs.

Self organization can be tribal and an experience of social Darwinism.

The Role of Leadership

When the Agile movement was born, it was common to have autocratic project managers who dictated what to do without explanation but then if things did not work out, somehow avoided the personal consequences. Those types of project managers were awful. Agile’s idea about self-organization no doubt was informed by experiences with such project managers, and was an attempt to take back control and responsibility for how programming teams did their work—that they should be able to work without interference from managers.

Yet, as I explained earlier, the Agile Manifesto did not throw out the notion of official leadership: instead, it said that when self-organization does work, it can work really well. Unfortunately, it often does not work well, and the Manifesto did not claim otherwise.

Much of the time, a team needs an official leader, but we don’t want the autocratic type; instead, we want a leader who is watchful but largely hands-off except when they need to step in; and we need a leader who acts like a catalyst, stimulating team members to be their best. This is known as a servant leader.

The Scrum community views servant leadership in a very weak way: a Scrum Master is supposed to be a servant leader, but the role is described as someone who is merely responsible for the team’s process, and who removes impediments. In my experience, much more is needed. The Scrum viewpoint does not see the Scrum Master as a contributor or decision-maker in any way. Yet, an effective leader often is one or both.

Many of the challenges that software teams have, especially in complex multi-team software efforts, is in identifying and talking through technical issues, leading to a timely decision. The Scrum view is that team members should each feel responsible for all of the issues, and collectively organize to discuss them.

But in my experience, those critical discussions do not happen sufficiently or effectively on their own, because the team members are each very preoccupied with their own stories. Exacerbating this, the issues sometimes span teams, and so team members often try to get their work done by focusing on what they can control—what is in the scope of their team.

The high frequency of technical issues is why there needs to be a technical lead who notices the issues that are not being addressed, generates discussion, and drives toward a decision. This is a Socratic approach.

Yet many tech leads are not good at the human aspects of that, and that is where a team lead needs to step in. But to do that, the team lead needs to have an interest in the technical issues that need to be discussed. Otherwise, they will tend to ignore them, and will not conduct a process for resolving the issues.

Face-to-Face Conversation—Sometimes Best, But Not Always

An important point here is that complex issues are not always resolved best through a single discussion or meeting. Sometimes it takes many discussions over time, with email exchanges, ideas expressed in a Confluence page, and whiteboard discussions as well. Working through complex issues takes time, and the process needs to be managed. Otherwise, it will take too long, or fall by the wayside in the rush to complete stories.

It is therefore essential that a team lead is sensitive to what kinds of interaction and dialog are needed for a given issue. For example, if some team members seem to be trying to discuss it “in email”, that might mean that written exchanges are important for the issue, so do not shut them down and force everyone to meet: they might not be ready.

An effective team lead needs to be able to help the team’s technical lead to orchestrate discussions about complex technical issues to reach expeditious resolution. Leaving it to the team is not effective. In my experience, if you leave it to the team, a large percent of critical issues will languish.

Summary

Self organization is a powerful concept. However, like so many good ideas, it becomes a bad idea if it is taken to an extreme.

Agile’s preference for self organization is just that: a preference, not a mandate. The idea of using self organization needs to be applied with judgment. The goal needs to be ensuring that a team operates effectively. That often requires overt leadership, but of a servant leadership and Socratic type.