October 12, 2018

We know why projects fail, we know how to prevent their failure - so why do they still fail?

I’ve been reading a lot about working on complex projects and most of it only touches upon the topic superficially with no clear guidance as to what steps to take and what to do in certain situations. So here are some of my personal observations and ideas (that are based on my own experience) on how to deal with building complex systems with multiple agents involved.

The easiest way to build a complex project is to reduce complexity. That’s even more true if the timeline is tight.

First of all, I consider complex projects those that are cross-product integrations with aggressive deadlines and multiple companies involved.

There are three skills that I consider being essential when working on such projects:

Adaptability

Be flexible, be ready for changes and try to see them before they even happen. Imagine the company that promised to do something for the project goes out of business. It’s highly unlikely, but it is a possibility. If this ever happens, it’s important to quickly adapt to the situation and find a way out, otherwise the whole project may be jeopardised. Or a simpler example: imagine some new law is accepted so the data storage needs to be redone for the project to stay compliant. Or an even simpler example: Imagine a person quitting, whose know-how was crucial to the project. These are all unlikely events, but I haven’t had one project in which not at one unexpected thing happened.

Failure is common, not the exception, in large-scale IT projects.

A more “developy” example: As different teams are working on different things, it’s possible that a change comes up during development either because something is literally impossible the way it was planned or it was decided that it needs to work differently because there’s a new stakeholder ect. This change may affect multiple modules which brings chaos into the development. Be ready for those.

Decisiveness

When something unexpected happens, it’s important to make decisions and to make them quickly. Don’t forget that all the decisions you make are your responsibility. So a good balance between quickness and thoroughness of your decision-making is key. The more time you take to resolve the unexpected situation, the higher the risk that the deadline will not be met. The quicker you make a decision, the higher the risk it will be the wrong one.

Communication

It’s as important to know stuff as it is to know which people to ask for advice. I consider this a skill and a very important one. Face to face discussions of the underlying problems without pointing fingers and putting the blame on someone else is crucial in almost every project. Even if someone from the project failed some part they were responsible for, it’s important to learn from it and move forward with new ideas and decisions. Too many times have I witnessed the passive-aggressive pushing of blame to avoid responsibility instead of accepting the failure and moving forward.

Best Practices #

Now that you know what skills you need, let’s talk about some of the best practices that will help dealing with complex projects.

Big Picture Meeting. Values discussion

Define the business problem you’re trying to solve, and the value in solving it. Leave the specifics, ignore the details. Talk about how it should work instead of how it most likely will work. Identify the stakeholders, maybe someone was forgotten. It’s important to envision a clear image of how the project should look like finished. For example, if the project is an airport, it’s helpful to not only talk about the runway, but also where the people will go to eat, how will they get there etc. If the project is a mobile application - a workshop for wireframes and the user flow will be needed.

“If you don’t know where you are going. How can you expect to get there?”

Building a team

It’s clear that you always need the best people working on complex projects. Make sure every area is covered with a person that will be responsible for that part. This person is then the key decision maker for that part and also the point of contact for anything related to that “module”. Usually there are several hierarchical levels involved so you form an “upper-team” and let those leaders select their own people.

Defining roles and responsibilities

Make sure everyone knows who’s responsible for what, who is able to make which decisions etc. Here’s it’s also important to define communication channel and intervals. For example, you say that every week on Thursday you will have a meeting to go through everything that was done and everything that went wrong. People need to be aware of the problems.

Research

Talk to your target audience, whether internal or external, so you fully understand the problem space and gather information on it.

Have a round table of workers from different background discussing edge-cases, things that could go wrong, ways to build it, ways it’s impossible to build and more.

Defining a Timeline

Start from the release date and go backwards. Discuss the interdependencies of the sub-projects. Does something need to be finished before the next phase starts. As an example: The content needs to be finished before the development starts, the design needs to be approved before development. The functionality needs to be signed off before the design starts etc.

Reworking

The project is not finished until the end user uses it. Be prepared for bugs at this stage. All the assumptions that were made, all the rules that were forgotten, ignored, jumped over will come to light after release.

To make this period less painful, you need to schedule some time before the main release for beta testing by the end users, a week, maybe even a month, depending on the release cycle and the size of the software. You will still get a huge bug list regardless of how well you test. This does not mean testing is meaningless. The more you test, the more bugs you’ll see and the more bugs you can fix.

No amount of testing can prove a software right yet a single test can prove a software wrong.

Most likely issues you will face #

Tight Deadlines

The business side always wants to have the highest quality in the shortest amount of time. As a person from the technical side, you always need to push back on that, as you know - the faster you go, the worse it gets. Shortcuts are taken, testing is forgotten, documentation is missing - the usual consequences of tight deadlines.

You either cut the scope and keep the deadline, or keep the scope and push the deadline.

No clear vision or conflicting vision

More often than not is that no one really knows what the vision of the project is. Or worse - there are conflicting stakeholders that are part of the project only because they had their own vision in mind and so the politic games will start. Consider an Ad platform that was built for several different (BIG) parties. Each of those parties considers themselves big stakeholders in the project so they will each favor their own vision.

Nobody would force a builder to build the basement after having put on the roof, but in the software industry, that is common practice.

Budget limits

We all want to work on multi-billion projects, with unlimited budgets. Sadly, reality really kicks you in the balls on this one, as the budget is usually the one thing that is tightly controlled. Everything that’s money related will need to be justified and the decisions will be under scrutiny.

The consultants/managers will try to minimize the costs so as to show that they are squeezing out the maximum efficiency out the teams.

Lack of project oversight

Delegation is cool, letting people run wild isn’t. If you give people the chance to slack off, they will take that chance. This doesn’t mean that you should micromanage everyone but you should rather define clear goals and make sure that people get there, without causing too much trouble.

For example, if some stakeholder is taking their time with some critical decision on which many teams are waiting, it’s important to step in and remind them that there are deadlines, and the project needs to be moving forward.

Vague requirements or gaps in requirements

It may happen that not enough homework was done during the planning phase (spec writing), so the requirements are not full and some small details were forgotten. If it’s really small, then the developer can make the decision himself how to fill the gap, if the detail is not that small or it’s a crucial part, then this will need to be handed to the relevant parties and will thus cost some time.

The larger the specs file, the less of a headache it will be for the developer.

Legacy code integration

Unless it’s a completely new project with no dependencies, you will have to work with some legacy system that wasn’t touched for ages and with which the integration will needed to be built. Sometimes you find some critical parts, that are relevant to your project, that weren’t touched for ages, with zero documentation and no one in the company that knows how it works. It’s better to evaluate the legacy system before the development starts, and to calculate the time it will take to build/migrate/extend the legacy code. (And add another several days, just to be sure)

Managers with optimistic view of the timeline

“If something can go wrong, it will go wrong” the saying goes. And this is very accurate in big projects. The problem with being optimistic is that if you think that each task will be finished at exactly the time allotted, then a single hiccup somewhere in the chain will results in everyone missing their deadlines. Each small mistake accumulates as an avalanche effect: Someone forgot to send an email, a decision is unmade, a manager is afraid of taking responsibility so they’re waiting for someone with more authority to step in, an angry phone calls here and there…

Change management is lacking

There needs to be a strict framework of how change requests are processed at all time. All relevant people need to approve the change before the execution starts. Yes, possibly the change request will take longer this way, but dealing with unexpected changes that come to light during release day is more unpleasant. If someone wants a change that is perfectly fine, as long as they follow the approved process and the change is approved. Don’t tolerate ad hoc changes from the team. These will invalidate the assumptions made before the project.

Best practice is to have a ‘change framework’ that needs to contain answers to 3 questions:

Who needs to be notified when a change is being planned? How do the people need to be notified? What needs to be communicated?

And the most important part, every change needs to be thoroughly documented.

Misconception that anything is possible in IT

People tend to forget the limitations of IT which results in engineers taking shortcuts to make things possible. If some module requires your network packets to come in faster than 20ms, but you are sitting in Europe and the Module is in China, this makes things a bit tricky.

In comparison to other engineering projects e.g. like building structures, no one will ask you to build a floating palace.

General Tips #

Document everything

It’s crucial to have everything in writing - all the information regarding who does what, who’s responsible for what, who built what, how the module works, how it should work or why it doesn’t.

Revisit plan at regular intervals

If you’ve successfully finished some part of your module - review how the assumptions that you made earlier changed. Make sure there are no issues going forward from this milestone. Talk to your team, do they see any issue now that some part of the module is complete?

Communicate progress in a timely manner

Set up regular reports with the key stakeholders so that you can tackle issues and manage expectations proactively. These could be written, or through any channel that was agreed upon.

Keep your options open

You don’t really know from the start in which direction project will evolve. So it’s impossible to prepare the core for any direction. Prepare to be flexible, but don’t start over-optimizing too early.

A megaproject contains a large proportion of predictable, standardised, and repetitive tasks that have been performed many times on previous projects — as well as novel and innovative procedures being applied for the first time.

Do your homework

It’s possible you’re building something you’ve never built before so it’s important to do research and talk with someone with know-how. When perform risk assessment, where are the pitfalls/shortcomings? Find out all the ins and outs of the project, what will affect the project as you move forward? How were similar projects built? Are there any case studies? This is essential to get your estimates right - the more you know the more accurately you will know what to expect.

Try and reduce the amount of parties involved

Multiple agents (subcontractors) always increase the complexity of the project. The best case scenario: your company can do the whole project alone, which sadly rarely ever happens.

The communication needs to be VERY STRICTLY controlled, otherwise it will get out of hand very quickly. Clear structures help everyone know where they sit in the hierarchy and who they need to talk to in order to get things done, for example e.g. change approvals, resources release.

Strict separation of phases

The content gathering phase should happen before any development begins. In complex projects there are more elements to integrate, some of which have high levels of uncertainty, such as requirements. There are also likely to be greater dependencies on other projects or external forces. In short, there’s a lot to pull together.

Frequently asked questions #

1. What’s the difference between software projects and other engineering projects?

The main difference is the amount of changes during the execution of the project. For example, if you’re building an airplane nobody will tell you in the middle of the project that they want it to go to space. But in software industry this is very possible.

2. How to estimate a big software project?

The task of estimation is a very important one, so take your time doing it. This is where good technical spec. document comes into play.

First of all, never estimate a software project by yourself, it’s good practice to have 3-4 people estimate the project and then take the average number of hours for reference.

Second of all, split the project into modules and modules into tasks and estimate the tasks, then sum the man hours needed for the tasks and for the modules. The rule of thumb is people only count the raw development time, forgetting all the small things that add up in between, so add another 20-30% for unexpected bugs, communication

will also take time: Scalability requirement, Security requirement, Performance etc.

3. Can I trust the developers to meet their own estimations?

This is a tricky one, if a developer estimates the task will take 30 hours, can you trust him to deliver finished work in time and up to the standards? To be honest you shouldn’t. If a developer says he will do this task in X amount of hours, you plan it as X + 20% hours internally.

The reason is that the developer may not see the whole complexity from the start and will only notice how hard it is after starting. I can not remember how often I started a task that seemed easy at first but was quite complex after the second glance. Consider a task to build a currentTime library that returns current time. Seems easy enough, but then time zones come into play, then suddenly some countries decide they’re not doing Daylight Savings anymore and the spiral continues.

In conclusion I want to say - never say no to a big project, it’s always a win-win situation. If you fail - you learn, if you succeed – even better. And never forget the importance of communication.

As always if you want to talk about this topic, send me an email, I’m always glad to chat and don’t forget to share!

24 Kudos