Agile DevOps

Breaking down silos

Learn how cross-functional teams are essential to successful DevOps

Content series: This content is part # of # in the series: Agile DevOps Stay tuned for additional content in this series. This content is part of the series: Agile DevOps Stay tuned for additional content in this series.

People at companies that create software products and systems frequently ask me, "How do we change our organization without actually changing it?" Of course, they don't use those actual words (at least not all the time), but that's the implication. They want to minimize the effort and risk involved in releasing software, but they realize it requires (cultural and technical) changes throughout the organization that they don't always have the power or influence to affect.

A cultural obstacle they often encounter is that traditional development and operations teams tend to work in silos, limiting the amount of inter-team communication until software release times. (And such communication is often confined to a series of tickets in an issue-tracking system.) Growing software enterprises must become more collaborative, or else they will cease to exist. The software industry is changing in this direction—much more quickly than some anticipated—as the result of cloud computing, which makes computing resources less scarce, and business demand. Companies that evolve will put those that don't out of business.

About this series Developers can learn a lot from operations, and operations can learn a lot from developers. This series of articles is dedicated to exploring the practical uses of applying an operations mindset to development, and vice versa—and of considering software products as holistic entities that can be delivered with more agility and frequency than ever before.

As emphasized in the introductory article of this series, collaboration across organizational boundaries is one of the anchors of agile DevOps. This article discusses how establishing cross-functional teams and broadening the skill sets of delivery-team members are ways to increase collaboration and break down the traditional barriers that prevent software from being delivered continuously.

The rise of cross-functional teams

A cross-functional team consists of experts across the entire software-delivery cycle, such as operations engineers, database administrators (DBAs), testers, developers, and analysts. Everyone on a cross-functional team contributes code to a version-control repository. For example, the operations engineer contributes infrastructure and configuration as code; the DBA contributes Data Definition Language (DDL), Data Modeling Language (DML), and datasets as code; the analyst contributes requirements as code; the developer contributes application code and configuration; and the testers contribute tests as code.

The role of specialists For new cross-functional teams made up of people accustomed to working in matrixed service-based organizations, it can take some time to become familiar with working in a cross-functional team. Often, the skill sets of individuals on the team are more narrow and need to be expanded. In this context, on the initial pilot projects, specialists in the organization (for example, security specialists) might be brought in to advise cross-functional team members. Advisors aren't part of the cross-functional team, and they don't contribute tests or code. Their purpose is to expand the skill set of the cross-functional team members. As it expands, the team will rely less on advisors.

Every member of a cross-functional team is responsible for the delivery process. Any person on the team can modify any part of the software system. The corresponding antipattern is siloed teams, in which development, testing, and operations have their own scripts and processes and are not part of the same team.

Cross-functional teams, then, consist of individuals from all the disciplines responsible for developing and delivering software systems. Instead of treating each discipline as a separate centralized service organization, the delivery team becomes the key organizational construct. Teams work together in a dedicated fashion to consistently deliver software, without the time impediments inherent when teams communicate across the organization. Consider composing each team of (at least) business analysts, customer representatives, DBAs, developers, project managers, and quality-assurance (QA) and release engineers. With a cross-functional team, you reduce "it's not my job" syndrome and other "walls" that stifle communication between teams both within and across physical locations.

Everyone doing everything

After some time, you'll find that skill sets often morph into more-complete engineers/analysts who begin doing more than just one part of the software-delivery process. For example, a programmer might create DDL and DML scripts. Or, business analysts define their requirements in acceptance-testing scripts (such as Cucumber) so that all changes run through automated testing based on the customer's requirements. Each team member knows that he or she needs to write tests, write scripts, version (see "Agile DevOps: Version everything"), and make everything they do part of a continuous system (see "Agile DevOps: Continuous delivery platform"). This is the case for all source artifacts: application code, configuration, infrastructure, and data. When team members become more polyskilled and silos are dismantled, communication is improved, and bottlenecks are removed from old-world software organizations.

The process

Everything that makes up the complete software system is either a script (that is, a program) or an automated test. Everything that makes up the software system is versioned, and everything is incorporated into a continuous-delivery pipeline. This way, when anything any members contribute is checked into the version-control repository, it creates the entire software system—including environments, database(s), and the software application/service using the versioned configuration. Every component of the system is versioned, with no exceptions. The members of the cross-functional team are 100% dedicated to the project and are affiliated with no other projects. This approach enhances communication and reduces any process bottlenecks.

How does it work?

Making cultural change in an organization is the most essential element of employing DevOps. Nothing else significant will happen unless this happens. This section discusses a high-level approach to transitioning to—and functioning effectively with—cross-functional teams.

Pilot projects

Attempts to change a large organization all at once are almost guaranteed to fail. The preferred approach is to start with a small pilot project that delivers something of business value to production within a relatively short time period—ideally, no more than 90 days. The pilot project should be something strategic—not an application that is rarely updated and delivering minimal business value. This new team will be a cross-functional team, and every member will be fully dedicated to the project. (They won't be working on other projects.) Every role required for developing the software system (operations, application developers, database, testers, and so forth) will be a part of this cross-functional team.

Service-oriented architectures To work in small teams that communicate effectively, you need to dismantle monolithic architectures. Architectures that have many other tightly coupled dependent systems are brittle and make change extremely difficult. This doesn't mean you aren't building large systems; it means that those systems are loosely coupled as services that communicate through a simple protocol.

Once you've proven success with one project, you can scale out to a few more. These other projects will have team members from the first pilot project who will share their knowledge and each be a 100% member of the new team. After these projects are successful, you will double or triple subsequent pilot teams until all strategic projects are operating under the new model.

Centralized responsibilities

Because all service teams are small (no more than 10 people) and independent, you might wonder which organizational functions are centralized. Although every organization is different, with teams that employ DevOps and continuous-delivery principles, centralized functions are either those that develop the platform services used by the rest of the software-delivery team or those that perform system monitoring (application, network, security, and so on). There may be release coordinators, governance, and management, as well, but none of these centralized teams should be performing activities that adds to the wait time of the cross-functional teams (in other words, the services the centralized teams provide are completely self-service). Often, in these organizations, the service-delivery teams (of programmers, operations, database, testers, and so on) are on call (usually this rotates between team members) and responsible for fixing production issues.

Self-service systems

You build it, you run it! When software-delivery systems are built such that only automated changes run the software delivery, anyone on the team (of no more than 8 to 10 members) is capable of deploying changes to production (or any other environment). As a developer (of infrastructure code, application code, analyst scripts, or whatever), your end customer is the actual user—not a traditional operations team—which encourages very fast feedback loops.

One of the key features of DevOps is that a team member should never need another team member outside his cross-functional team to perform an activity as part of the delivery process. Everything must be self-service. Any team member should be capable of delivering software to production (see the "You build it, you run it! sidebar"). When team members are designing any feature in a software-delivery system, they need to consider how to develop it so that it can be used without delivery pipeline wait times involving submitting tickets, sending email, or using other communication mechanisms. (These tools are often still used with teams who embrace DevOps and Continuous Delivery. But in the context of the delivery pipeline, they are automated.)

Patterns

With the overarching principle being to move all internal services toward self-service models, certain specific patterns apply that are covered in one way or another in this series so far. The most essential of these patterns are described in Table 1.

Table 1. Key patterns that support DevOps work

Pattern Description Big visible dashboards Teams across the organization get real-time information on the state of the software system, including build status, customer metrics, and availability. Colocation Teams are in physical proximity to one another to enhance communication. Continuous integration You build your software (environments, applications, and so on) with every change. Cross-functional teams Software-delivery teams are composed of experts across various disciplines, including programmers, testers, analysts, and operations. Polyskilled experts Reduce specialist silos by expanding skill sets on cross-functional teams. Scripted deployments Deploying software to environments is entirely scripted so that it can be run from a single command. Scripted environments Creation of environments is entirely scripted so that it can be run from a single command. Self-service releases ("You build it, you run it.") Any authorized person on the team can and does perform deployments to production. Stop the line Anyone can and should stop the continuous-integration system when necessary. Test-driven everything Write automated tests for everything: application, infrastructure, everything. This might include writing unit, acceptance, load, and performance tests. Version everything Version all artifacts: infrastructure, configuration, application code, and data.

The death of old-world software development

Get involved developerWorks Agile transformation provides news, discussions, and training to help you and your organization build a foundation on agile development principles.

You learned from this article that one of the keys to effective DevOps is breaking down silos and creating cross-functional teams that can deploy their software to production. Organizations that have long procurement and delivery cycles will either evolve or go out of business. In large organizations, evolving can take time and requires a change in organizational culture.

In the final article in this series, you'll learn how you can create a DevOps dashboard—a comprehensive view of a software system's state for development and operations teams to monitor in real time.

Downloadable resources

Related topics