Software engineering is hard and easy.

Creating software has become easier over time. Yet software engineering is evolving with its own challenges. The first step to solving a problem is understanding it. Understanding the hard parts is the first step to making it easy.

There is programming and there is software engineering. Software engineering is not just programming.

First, the easy parts: We have more software engineers than ever before. Access to global teams has expanded. Tools, computing power, infrastructure, technology, and learning materials have never been this readily available and accessible. There are considerable improvements in the software development process. We have access to many programming languages, techniques, designs, algorithms, patterns, and use cases. Technology is advancing very fast, and so is the adoption. Building a software product is getting easier and cheaper, and so is owning software. For one dollar, you can afford an app with unlimited upgrades. It is easier to use software than ever before. A four-year-old can choose and play his or her favorite videos.

Now to the hard parts. We have more software engineers than ever before, yet we struggle to hire and strive to retain. We still struggle working late nights to manage an underestimated project and spend sleepless nights debugging someone else’s code. We take hours or days to fix a broken service. Companies struggle to manage software budgets. We still build software that hangs, loses data, becomes inaccessible at peak times, and kills companies. We find it is hard to scale — software, teams, and customers. We still know a lot of security compromises taken, performance tradeoffs, known bugs, and yet ship it to production. There are millions of lines of code in production, transforming industries, which is a nightmare to maintain. And we know it is going to stay there for a few more decades.

Software engineering is hard and easy.

Parts of software engineering

If we decompose software engineering, there are four main parts — problem, software, people, and process. And there is a bonus; all these parts are bounded by business context.

When we have

- a problem (what to and what not to build)

- solved by people (the hardest part)

- building software (solution)

- by collaborating (process)

- bounded by business context (dynamic rules of the game), we call it software engineering.

When there is only one problem, one person, there is no process required — software engineering becomes easier. Then why should we add people? Mostly because of time. We need to catch up with the market needs on time.

Scaling these parts pose a separate challenge. When the problem scales, we struggle to make choices and prioritize; when people scale, it becomes hard to decide and collaborate; when software scales, it becomes hard to maintain. It is not impossible, but it is hard.

1. Problem:

“Software is eating the world.” — Marc Andreessen. There are so many interesting problems. From product to features, ideas are abundant.

“Success is not delivering a feature; success is learning how to solve a customer’s problem.” — Scott Cook.

Understanding the problem is not straightforward. It involves guesswork. It requires understanding and agreeing on what to do and what not to do. Hard parts include identifying the problem, estimating, articulating and narrowing down the choices, experimenting, accepting the changes while building, being flexible with the plan, and delivering on time.

2. Software:

Software is the solution to the problem. The hardest part of the software is that it does not have a physical shape. It limits the ability to understand as complexity increases.

Two people can see a car and can agree on how it is. But the same two people cannot take a code base and view it the same way. Two people cannot write the same code for the same problem.

There are a few constants and lots of variables. And generally, the software is not built from scratch.

3. People:

src:holocracy.org

“Software engineering is what happens to programming when you add time, and other programmers.” — Russ Cox. People are the hardest part of anything, let alone software. Most of the people in software come from a mathematical background. And they assume that people are rational and work in the way math works. If one person does a piece of work in twelve days, the same task can be done by three people in four days. That is what math teaches us. Well, it does not work that way. First, we find it hard to accept it because it questions our mathematical skills. Then, there is the experience. And different people have different experiences.

People come with different strengths and weaknesses. It is hard to assume that they only play on their strengths. Someone who is a superstar in one team or company may be a mediocre performer in another. And two superstars may not work well together.

4. Process:

1950: Just 67 Seconds after he stops. 2013: 2 seconds. Src: https://www.youtube.com/watch?v=RRy_73ivcms

Process is the agreed way to collaborate. It is supposed to remove ambiguity and increase velocity. Bees collaborate, ants collaborate. They do it in the same way forever. It is hardcoded. If we take two groups of ants (of the same type) and give the same challenge, they may not show considerable difference in finishing the task. If we do that with people, there are chances of a significant difference in the time taken to complete the job. It is where the process comes in. This is where diversity helps. Bees and ants don’t argue about the process. People do. They need to decide, and decisions take time. Process is supposed to make things easier, but many times, it adds overhead.

5. Business:

Business is the dynamic rulebook for these parts. This is the most irrational part. All other parts are bounded within this. A sudden market condition can block you from offering after a painful hiring process. A new compliance law can halt your feature development and make you focus on restructuring the data. A competing product can make your product or even the company obsolete. With a lot of startups in the last decade, there is hyper-growth. And sudden hyper-growth attracts chaos, and many times, an addition to the team is more of a hindrance than growth.