Tech Debt

Not taking appropriate action to retire old systems, rewrite and/or move existing legacy systems

At some point, a legacy monolith reaches leviathan proportions, and is such a tangled web of disaster that it harms development. Existing veteran developers have trouble working with it, new developers strafe in the dark morass for months before being marginally productive, and strange crashes lurk causing unknown issues for your clients. But the management keeps prioritizing product, not realizing that the tech debt largely slows the development of the product features they so desire.

Solution: Business must yield priority to tech debt. The most egregious issues with existing legacy must be prioritized and addressed. Longer-term rewrites must be sized and handled over time. This will pay huge dividends in development time of features, and developer happiness and longevity. It’s often said that developers don’t quit their jobs, they quit their bosses, but I think many times, developers also quit their codebases.

Writing new systems that don’t need to exist simply to scratch an itch to write (NIH syndrome, etc.)

“Have you heard of that cool new programming language? I’ve been learning it on the weekends and I want to see how it works in production. I’ve heard great things.” Time passes, and a new system is built in a new language, that does something very similar to an existing open source library, only in a more esoteric use-case-specific fashion, and with fewer developers to support it. Sooner or later, the project decays, the developer leaves the company, and much time was wasted on this system that wasn’t necessary which could have been used for business-critical work.

Solution: This one is tricky. It takes a seasoned manager to know when to red-light this kind of project in favor of using existing solutions, and when to green-light things. You don’t want to suppress good developers, but you don’t want waste. The best that can be done is to ask the developer to write a thorough proposal that sells the specifics of the implementation, and to thoroughly vet it. This is, alas, one of the pitfalls of hiring less-experienced managers.

Blame the legacy, it’s all the legacy’s fault

Ah legacy systems. Everyone loves to swear at existing code, blaming developers who often are no longer there and can’t defend themselves. Things must have gotten this bad because they were awful, but we’re better. Sure we won’t fix what’s there, and our added hacks were necessary, but we’re better, really, we swear. It’s so much easier to build hacks on top of other hacks, it really makes you wonder how the legacy got here in the first place. This creates a toxic environment of bottomless pits of complaints, and ultimately hurts code quality and team happiness.

Solution: While it may be tempting to endlessly complain about existing systems which may indeed be suboptimal, take the time to remember that the existing code brought the company this far, and that dubious design decisions may be the result of several changes of the guard and business decisions combined with emergent disasters which surround any fast-growing business. Is something awful? Make a concrete plan to make it better. Do it, even if you just chip away at it. The person who comes after you (it might be yourself) will thank you for it.

Design

If you spend more code to work around the weakness of a system than the advantages it gives, this is bad design

Someone has a great new design they want to implement which is leagues better than what exists, and the existing design is just so awful. Only, it’s going to require some duct tape here and there, but that’s totally acceptable because overall it’s better. Until it becomes apparent that it’s mostly duct tape, but you’ve gone all-in on this and you can’t go back.

Solution: Start over. The sunken cost fallacy is real. You’ve made a mistake, and that’s ok, you can take those learnings and build something better. Don’t settle for chaos.

If most people working on this codebase run into the same set of problems avoidable by a different architecture, this is bad design

You notice the same piece of code gets copy+pasted everywhere this system is involved. You ask someone why, and they explain “oh, that’s because this system expects this thing for a reason that isn’t actually true but sounds good, and everyone has to play along.” You have no idea why they did things this way, but you’re busy, you paste the snippet into your own code and move along.

Solution: It’s painful to hear, but, refactor it. Refactor the whole thing and get rid of the copypasta. It will pay dividends.

Coding

Shotgun coding. No tests, or bad tests. Not handling of corner cases, mainly happy-path coding.

“This is a startup, we don’t have time for this, we’ll handle it later. This is acceptable,” said the many developers over the years, each contributing their own piece to the chaotic mess that this became. Push out code with no tests, break production and fix it later, write tests to handle only the happy path, or not even that.

Solution: Management can enforce test coverage, but really this is not a good measure of how well-tested a codebase is. What does work is forcing code reviews by thorough people who check for these things, and calling people out for doing superficial code reviews. Unfortunately, this is a case where hiring less experienced or conscientious engineers will hurt you badly, so watch your hiring for this.

Meetings

No agenda, or fluffy agenda with no measurable output

You check your calendar and find you’re in a meeting called “Discuss The System” with no description, 10 people invited, and it’s 1 hour long. Being the good employee you are, you show up, stay silent through the majority of it as 2 people ramble on, and at the end of the meeting, the organizers pat themselves on the back and end it, and threaten to hold another one. Time is wasted, irritation brews, and nothing has been gained, but much has been lost.

Solution: Enforce a company-wide policy that all meetings must have an agenda with a measurable output, involve only the necessary people, and have notes taken to show if the output was significantly reached. End all meetings once the goal has been reached, or the time is up. Any meeting that doesn’t obey these tenets, allow people to either leave, or not show up in the future. This will force people to change.

Meetings that could be an email or a DM

You’re in a meeting with an agenda with measurable output (yay!), but it’s a simple yes/no question that could be answered through email or DM.

Solution: Yes, this is a shocker: use email or DM. Use your common sense, and use the appropriate medium.

Meetings that include unnecessary people, and don’t include necessary people

You’re in a meeting where 10 people are present, except the main designer. People wave their hands and say words, but in the end, all of this will just be overridden by the person responsible. Later that day, you’re in a meeting where for some reason the head of engineering is present to go through a low-level code decision, and nitpicks things out of context, wasting everyone’s time (including her own).

Solution: Have a good reason to invite every single person at the meeting. If you can’t think of one, don’t invite that person. If the chief decision makers of something should be involved, invite them. And above all, stop adding people as “optional” to a meeting. If a person is optional, they should not be invited. It’s that simple. This is a company meeting, not a tech conference seminar.

Meetings had for the sake of meetings (standups, what are you really gaining from this?)

You’re at the daily standup meeting. People talk about things that you have no idea about, nor care about. You likewise share what you do that they don’t have any connection to. The few people who do have connections, you’ve talked offline to already, and here you report that you talked offline to them, and will continue to talk offline to them. Wait, why are we here again?

Solution: If you absolutely must know what other people are working on, write it in text in a chat or email. “Oh, but nobody will ever read that!” you say. Correct. That’s because nobody cares. Do you think that hearing it aloud will make them care any more? Are you all now doing a better job as a result of this daily ritual? Probably not. Stop wasting people’s time, and focus on the results. Kill all useless meetings, especially the repetitive ones that are held mainly for ritual.

Meetings too early or late in the day. Attention span low, irritation level high. You won’t get good things from these meetings.

It’s 6PM and someone’s holding a meeting. You latch onto your third coffee of the day and pray this is either good or ends soon. Neither of these things come to pass. The next morning you realize someone has put a meeting at 8:30AM. You latch onto your first coffee of the day that you haven’t half-finished and hope this is good. Someone asks a yes/no question, and the meeting ends. Seriously?

Solution: Seriously. This is one of many ways in which we’ve forgotten the human factor. Stop scheduling meetings too early or too late, schedule them during core hours. If a meeting is so important to have, it needs to be done immediately. Otherwise, there’s nothing that can’t wait a day or a few hours. People will be more willing to be present, participate more, and you’re more likely to achieve the desired outcome.

Process

Blind adherence to process without evaluation for current situation

Sprints! Who doesn’t love sprints. We do Agile here. What do you mean Agile isn’t a noun? I just capitalized it, it must be so. Anyway, let’s just repeat sprints every week or so, and have people figure out what they’re doing. How do I know this is working? Because we create another sprint every week. What, results measurement? Many tickets were created and closed. What more do you want?

Solution: Process is a great thing to have. It guides work, and leads to desired outcomes on the roadmap (which you of course have!). However, process that does not measurably get you closer to or accomplish your goals is either unnecessary, wrong, a waste of time, or all of the above. Always evaluate your process for what it gives you vs. what it takes away, and ensure the former is always greater.

Excessive democracy leading to indecision

Everybody’s opinion matters. We’re not a dictatorship here, ok? Let’s ask for comments from everyone in all of engineering. Wait, two people are disagreeing, that’s not good. One of the engineers isn’t in any way involved in this project, but she has great experience! Until this is settled, we can’t move on. Forget the deadline, we need to do things right! Correctness matters.

Solution: No, corporations are not governments, and they should not be democracies. Fast, good decisions save a company, and slow decisions, good or bad, can kill a company. If you assign someone to be in charge of a project, they get the final say, and they make the decision quickly. Disputes can be resolved more privately without bringing in the entire engineering department. Focus your people, and you’ll maximize your outputs. Give them all a say in everything, and you’ll be gridlocked for no good reason.

Excessive micromanagement, manager not taking care of bigger picture, ICs frustrated, bad work comes out

The manager is a people’s person, she’s right there with you in the trenches. She asks you about this line of code and why you didn’t handle it another way, not knowing the context that led to the decision. You spend an hour explaining it to her, and in the end she goes “ohhh” and agrees. Meanwhile, the roadmap is running late and is not accurate, engineers are angry left and right, and the manager is nowhere to be seen on handling these real issues.

Solution: Trust people. If you can’t trust them, get rid of them. Micromanagement is the death knell of any organization, and stops any team from scaling. If the manager isn’t handling real high-level decisions, that one line of code won’t matter, because the house is on fire while you were arguing with the carpenter about the new chair. Stay above it all, use the bird’s-eye view to make great decisions, and even if small inefficiencies arise at the lower levels, the ship is still sailing in the right direction.

Hierarchy that hinders but is not used. Managers skipping rungs on the ladder to ask non-direct reports for updates.

You’re working on something, and your boss asks how it’s going, so you give an update. Then your boss’s boss asks, and you give the same answer with less detail. Then her boss asks, and you say “good.” Then a manager completely unrelated to your team asks and you begin to wonder what’s going on here. You decide to just write a company-wide accessible report of what’s going on, wasting your time to give excessive visibility to managers who should have other concerns. What’s going on?

Solution: As always, corporations are not general society. The hierarchy isn’t there to suppress the lower class, it’s there to maintain order and efficiency. If a manager 3 levels up is asking an indirect subordinate about a project regularly, it’s a sign of micromanagement, or too many managerial layers. Both problems are obvious how to address.

Lack of useful checkpoints

We’ll know the project’s done when it’s done. We report status in number of tickets completed. Sure, the number of open tickets varies over time, but it’s a rough idea, right? Until the project that was scheduled to take 2 weeks takes 6 months.

Solution: Make checkpoints. “Such and such is expected to be done at the end of week 2.” If a checkpoint is not met, estimates must be reassessed, and possibly more people must be allocated to a project if it’s high priority. Otherwise, you have uncontrolled ballooning of a project’s scope and time taken.

Not stepping in to help an engineer when they’re stuck

Everybody knows that Jackie’s working on this long-term project, and that it’s taking a while. You often wonder how she’s doing. Management doesn’t seem to care she’s delayed, because hey, it’s just a difficult project you know? Time passes by, and nobody steps in. She asks for help from time to time, but she’s told that everyone’s busy. The project takes forever.

Solution: Management should have some visibility on the status of projects. If something appears to be taking too long, then at some point, it’s wholly appropriate for management to step in and figure out what information is missing, or if more headcount is required. This is not micromanagement, this is a core part of management, but oftentimes managers who fear micromanaging give so much freedom as to not manage at all. It’s admittedly a tight line to walk, but this is the hallmark of good management.