Refactoring is the cure for Technical Debt — but only if you take it
Technical Debt — a catchy new name for a problem that product/project managers have known about for ages. What it means — over time, all software becomes inviable without significant re-investment.
Like human mortality, technical debt is inevitable. And much like the causes of human mortality, naming a problem is the first step in the course of understanding its causes, symptoms and cures.
Technical debt accrues after managers/teams make hard decisions about time and resources. We don’t have unlimited resources, and we don’t have unlimited time. Projects have a budget and a deadline, software needs to be shipped.
Conversations about technical debt are necessary at all stages of a software project. Technical debt grows on an exponential scale — all we can hope to do is delay its initial rate of accumulation and minimize the chance of significant rate-inflection events.
At the concept stage, good design (UX/architecture) is the best approach to limiting a project’s future technical debt. Performing a thorough review of user needs, available technology, and market/tech trends is critical. On the architecture side, abstraction and modularity are your best weapons. When one part of your system requires improvement (e.g. you need a mobile app that contains the functionality of your web app), modularity will save your ass (e.g. a robust and independent server API).
The API revolution, and its older relative, three-tier architecture, has rescued many a software system from an early demise. For example, systems that use hard-coded connections from the front-end to the back-end (e.g. SQL queries, two-way data binding) are screwed as organizations migrate to new data warehouses. I’d go so far as to claim that’s a big reason why SQL has survived the NoSQL onslaught — too many legacy, business-critical systems aren’t modular enough to allow change in their back-end SQL interface.
This article is about refactoring — the only thing you can do to reduce technical debt after the project has already been designed and started. Most managers and developers never have the luxury of starting a project from scratch. The design phase has already taken place (likely without your input) — and technical debt has already started to accrue.
There is a natural resistance to refactoring from most people involved in a software project. Why?
- Refactoring seems like waste to the “un-woke” project manager. It doesn’t add any new desired functionality, and doesn’t directly fix any bugs. It doesn’t match anything on their priority board.
- A “woke” project manager can evaluate how a refactor of certain components could significantly reduce the risk of future technical debt. But often they don’t have the ability to forecast if/when the refactor could be completed and how much it might cost. Too risky to justify to the higher-ups, given resources and deadlines.
- Junior-level developers won’t even consider refactoring, either because they don’t have the perspective to see a better (abstract/modular/scalable) solution, or because their performance review is too closely tied to that project manager’s priority board.
- Some senior-level developers who have been involved in a project from its early stages have too much pride/investment with the existing system, and their life will be much easier if nothing changes. In fact, their power within an organization often increases with the technical debt of the software (at least until it’s killed), because only they are able to make patches/fixes/updates to their lumbering contraption.
Agile methodologies for software development include refactoring as one of their core concepts. You may note, however, that nothing in the 4 scenarios above was particularly tied to any project methodology. Whatever the project development system, the old adage will always apply: “if it’s not broke (according to the current specification), don’t fix it”.
The problem there is “according to the current specification”. Current specifications are the driving force behind technical debt. On the other hand, keeping to a current specification (either fixed or mutable/agile), is often the only way software gets shipped.
That’s the multi-million dollar question — ship on time/budget or prepare for the impending/changing/scaling future? There’s no easy answer — and every case is different.
The only thing an organization can do is to recognize the significant mid/long-term value of refactoring and give project managers/developers proper incentives to do it. Some guidelines:
- Sometimes a refactor is as simple as going back and renaming/organizing classes, methods, variables and tests. Good code is self-documenting —the names of things should describe their purpose.
- Encourage developers to do minor refactors at all times, even if it puts development/sprints slightly behind schedule. Minor refactors include changes to code that implement best-practice design patterns. This also means that developers must be aware (at hiring-time, or with ongoing education) of best-practice design patterns.
- Listen to and reward developers/designers who have an idea for a “better way”. It might stall the project or increase the development time — but if the “better way” increases modularity within the system, it could be a major win.
- Consider early branching/forking of the codebase, and grant developers the necessary side time to test out that refactor while still contributing to the original branch (current specification).
- Balance the short-term pressure to deliver working software (current spec) with projections of technical debt rates after completion. Not all refactoring needs to be done before the MVP or version 1.0 is released.
- Premature optimization is the root of all evil. This is a common technical-contrarian argument against refactoring. But premature optimization is often the opposite of refactoring — it’s a doubling down on the status quo.
The simplest thing to understand about technical debt is that the cost of a software project does not stop once 1.0 is shipped. Systems with high technical debt will cost much, much more afterwards. Good design at the initial stage of a project, and a refactor-friendly culture are your only weapons against this threat.
Best of luck!
