2. Monolithic Development Habits

Communication structures, teams, operations, deployments, and processes all look different in an SOA world. The tools we’ve gathered over years of monolithic development aren’t necessarily the ones to use when working with services. Old tools/habits need to be replaced by their SOA equivalents if developers (and the organization) are to succeed with SOA.

Local development

When working with a monolith, we run most (if not all) infrastructure on our local machine. Developers, at one point or another, have run into the “it doesn’t work on my machine” problem. Organizations invest in bootstrap solutions to avoid this problem and make developers productive as soon as possible. Most solutions are centered around making the infrastructure easier to run on your local machine.

Running everything on your local machine doesn’t scale when you start working with multiple services.

In most cases, dev machines aren’t powerful enough to run the necessary services during feature development. You may get away with running a handful of large services, but there will come a time when your machine won’t keep up.

Running services locally means developers have to know how to run (and potentially deploy) a service they don’t own. Ideally, service owners shouldn’t need to worry about any service outside their ownership.

Investing in solutions that make it easy to run multiple services locally is the monolithic mindset, and approaching service development the same way hints at a distributed monolith. I’ll leave some resources at the end of the article on how other companies approach local service development, but here’s what those solutions typically look like:

Mocking interactions with other services using dependency injection and client libraries.

Running some of your infrastructure in the cloud, running some of it locally, and plugging your local infrastructure into your cloud infrastructure.

End-to-end testing

There are two forms of end-to-end testing:

Automated tests, which are usually run in CI/CD pipelines.

Manual tests, which developers do before checking in their code and while they’re doing code reviews.

End-to-end testing in a monolithic codebase is somewhat straightforward. After setting up the appropriate data/state, you’re able to test the new user flow on your local machine.

Even if we write good unit/integration tests that verify the correctness of code, manually testing everything on our local machines during QA increases our confidence. As your service ecosystem grows, features and use cases will be backed by multiple services and it will be infeasible to test everything locally:

Setting the appropriate state may take a long time.

It may be hard to mock interactions with non-trivial systems (message brokers, asynchronous job queues, etc.).

Your machine isn’t capable of running the required infrastructure.

End-to-end testing looks much different in SOA. Pursuing a solution that makes it easy to do end-to-end tests locally is expensive and doesn’t scale.

SOA is popular because of its promise to quicken iterations. If you’re spending a considerable amount of time testing your changes locally, you’re not positioning yourself to take advantage of SOA. At some point, you’ll need to let go of the psychological safety of testing everything locally.

Your testing strategy depends on the local development solution you’ll adopt. Here are some ways to release code without having tested user flow locally:

Hiding new features behind feature flippers, allowing you to test user flow in production before enabling the feature for everyone.

Canary deployments, shadow deployments, red/black deployments, etc.

Stress testing new changes in staging before rolling out to production.

Of course, the psychological safety you surrender by dropping local testing can be procured in other ways:

Thorough and well-thought-out instrumentation/observability.

Alarms, workbooks, and rollback procedures to help you recover from failures.

I’ve listed testing/deployment strategy resources at the end of the article.

Debugging

On the flip side of the same coin, if your testing strategy needs to change, so does your debugging strategy.

In monoliths, a stack trace is enough to get us started. A stack trace points us in the right direction and we follow the cookie crumbs until we reach the problem. Stack traces and traditional debugging tools are usually enough to debug issues in SOA, but they’re useless for certain classes of issues:

Transient network errors.

Data sync issues across multiple services.

Incorrect configurations — connection timeouts, read/write timeouts, number of workers, scaling configurations, etc.

Although you’ll still comb through code to find the issue, your debugging toolbox should also include: