Lessons learned from a big software migration & reducing technical debt

How we successfully migrated our application without breaking anything

Major migrations in software engineering can be challenging. I was part of migrating a more than two-year old Angular 4 application used daily by many big companies to the latest Angular 7. First, I will talk about rather general steps you should take which do not solely apply to Angular migration projects. Then, I will go into more detail how we followed these steps.

But what are the reasons for doing these migrations. What about “Never change a running system”? While this is not untrue, I believe a running system may not be enough. An application may still work and existing users keep paying for it. On the other hand, there are several reasons against standing still:

Your competitors do not sleep : they will improve and potential customers will likely prefer a product which is still evolving.

: they will improve and potential customers will likely prefer a product which is still evolving. Hiring new talent is essential for any ambitious company. An outdated technology stack may turn off candidates.

is essential for any ambitious company. An outdated technology stack may turn off candidates. IT Security is a prevalent topic. Many security incidents occur due to known vulnerabilities in third-party libraries. Thanks to the open-source community, known security vulnerabilities are often patched in a matter of days.

is a prevalent topic. Many security incidents occur due to known vulnerabilities in third-party libraries. Thanks to the open-source community, known security vulnerabilities are often patched in a matter of days. It can become more difficult to move fast and break things . Although you should not break things, the idea is that if your tech-stack is stagnant, it might slow you down in your work. For example, an outdated tech-stack could prevent you to move as fast as your competitors.

. Although you should not break things, the idea is that if your tech-stack is stagnant, it might slow you down in your work. For example, an outdated tech-stack could prevent you to move as fast as your competitors. An outdated technology stack makes developers life harder. Popular third-party libraries or important bug fixes may require up-to-date frameworks. Besides, if your framework version is severely outdated, developers will become lazier to try out new things: “Hey, Angular Universal could improve our performance by a wide margin! Well, too bad, we are still on AngularJS and cannot use it.”

You may ask yourself: “Well, how hard can an upgrade be? Just run npm update or something and you are good, right?”. The reality is that many tech migrations (not limited to tech) fail or take much longer than expected. Some possible reasons why migrations can be hard to do:

The technology has changed drastically requiring you to change lots of things like tests, dependencies etc.

requiring you to change lots of things like tests, dependencies etc. Missing migration guides or insufficient changelogs make it harder to understand what has been changed and what actions you need to take in order to upgrade.

make it harder to understand what has been changed and what actions you need to take in order to upgrade. The goal is not clear . If you create new construction sites while doing the migration, the process will take longer and become more complicated to manage.

. If you create new construction sites while doing the migration, the process will take longer and become more complicated to manage. Missing knowledge about the used tech-stack makes updating more complex.

What are the general steps to migrate?

For frameworks, there are often update guides when a release contains breaking changes.

when a release contains breaking changes. Build tools like Gradle for Java projects or package managers like NPM for JavaScript projects make it easier for developers to manage dependencies. Run npm outdated to see which dependencies could be updated. Especially third-party libraries may have been updated in order to support the latest version of a framework. Before updating, I recommend to check your third-party libraries to see if they actually support the version you want to update to.

to see which dependencies could be updated. Especially third-party libraries may have been updated in order to support the latest version of a framework. Before updating, I recommend to check your third-party libraries to see if they actually support the version you want to update to. Check the changelog for breaking changes . Breaking changes are usually highlighted by the library author in order to let users of the library spot them more easily.

. Breaking changes are usually highlighted by the library author in order to let users of the library spot them more easily. Automated Testing & Continuous Integration can help you identify issues before releasing. This involves doing a full production-ready AOT build as well as running both the unit-tests and the end-to-end tests. This reduces the need for manual testing and allows you to ship code with more confidence.

can help you identify issues before releasing. This involves doing a full production-ready AOT build as well as running both the unit-tests and the end-to-end tests. This reduces the need for manual testing and allows you to ship code with more confidence. Manual Testing is still something that should happen even though automated testing should reduce the need for it. Some changes like untested functionality or style differences may not be caught by building or automated testing. The manual testing should not only be done by the developers but also external people (e.g. people from QA) because they can spot issues developers could miss. Manual testing should only be done once automated testing is passing.

is still something that should happen even though automated testing should reduce the need for it. Some changes like untested functionality or style differences may not be caught by building or automated testing. The manual testing should not only be done by the developers but also external people (e.g. people from QA) because they can spot issues developers could miss. Manual testing should only be done once automated testing is passing. Comparison of the application before the update vs. the application after the update allows you to identify behaviour that changes during the update process.

allows you to identify behaviour that changes during the update process. Improved tooling and a more mature ecosystem makes updates easier . Powerful CLIs, official documentation, training material and a vibrant user base improve the developer onboarding process and the experience.

. Powerful CLIs, official documentation, training material and a vibrant user base improve the developer onboarding process and the experience. Big-bang updates can be problematic . Instead, you could do a spike: take one/two days of work to estimate how much effort is required.

. Instead, you could do a spike: take one/two days of work to estimate how much effort is required. Do not let the migration branch lay around too long . Since other people will probably change code as well, it will be more difficult to resolve merge conflicts the longer the migration is taking.

. Since other people will probably change code as well, it will be more difficult to resolve merge conflicts the longer the migration is taking. Do not mix other tasks with your migration task . Focus on finishing the migration before doing other unrelated tasks.

. Focus on finishing the migration before doing other unrelated tasks. A features overview what your application can actually do makes testing easier. You do not have to create a big document: good test coverage helps a lot to identify features of your application.

In the next section, I want to focus a little more on how we did these things.

How did we follow these steps in order to migrate?

The Angular Update Guide is a really big help. It shows you the general steps you need to follow in order to update Angular. We followed the instructions in the Angular update guide which includes the essential steps and also mentions breaking changes.

is a really big help. It shows you the general steps you need to follow in order to update Angular. We followed the instructions in the Angular update guide which includes the essential steps and also mentions breaking changes. We created a diagram which shows the update path in more detail. Instead of a big-bang update, we tried to incrementally update.

which shows the update path in more detail. Instead of a big-bang update, we tried to incrementally update. Through scanning through the changelogs of our third-party libraries, we identified some breaking changes in advance which required us to adapt our code. An example was RxJS which had introduced breaking API changes in version 6.

in advance which required us to adapt our code. An example was RxJS which had introduced breaking API changes in version 6. Seeing which dependencies were outdated or not needed (anymore) at all allowed us to see that there were no dependencies which would not work in Angular 7 . Especially third-party Angular libraries (e.g. ngx-bootstrap) may have been updated in order to support later Angular versions. Fortunately, there were no dependencies which would not work in future Angular versions. Some third-party libraries needed to be updated. This was also an opportunity to remove third-party libraries which were not used at all or to replace them with more up-to-date libraries.

. Especially third-party Angular libraries (e.g. ngx-bootstrap) may have been updated in order to support later Angular versions. Fortunately, there were no dependencies which would not work in future Angular versions. Some third-party libraries needed to be updated. This was also an opportunity to remove third-party libraries which were not used at all or to replace them with more up-to-date libraries. We have a good unit-test and end-to-end testing coverage which helped a lot to ensure we did not break functionality. .

which helped a lot to ensure we did not break functionality. . For manual testing, we asked other developers as well as customer support people to participate at our “QA day” . For a few hours, we asked the participants to scan through the features we have and to look for issues. This way, we identified some smaller bugs which were quick to fix.

. For a few hours, we asked the participants to scan through the features we have and to look for issues. This way, we identified some smaller bugs which were quick to fix. After some major changes, we compared our current version with the old version to ensure we did not alter behaviour. Also, the comparison of old vs. new uncovered smaller style issues which were easy to resolve.

to ensure we did not alter behaviour. Also, the comparison of old vs. new uncovered smaller style issues which were easy to resolve. With the newest Angular CLI, the update process got easier through the usage of ng update . Since Angular CLI v6, there is a quite handy CLI command called ng update that can be used to update your application and its dependencies.The jump from Angular 6 to Angular 7 was done in a few minutes compared to the jump from Angular 5 to Angular 6. Also, Angular now supports custom webpack configurations so we could get rid of workarounds.

. Since Angular CLI v6, there is a quite handy CLI command called that can be used to update your application and its dependencies.The jump from Angular 6 to Angular 7 was done in a few minutes compared to the jump from Angular 5 to Angular 6. Also, Angular now supports custom webpack configurations so we could get rid of workarounds. During the Angular update, we noticed some improvements that were nice to have but not mandatory . Instead of creating even more construction sites, we collected them and focused on finishing the migration in a tidy manner first. After the update, we could tackle those improvements without worries since the main task had already been taken care of and these improvements were more like ice on the cake.

. Instead of creating even more construction sites, we collected them and focused on finishing the migration in a tidy manner first. After the update, we could tackle those improvements without worries since the main task had already been taken care of and these improvements were more like ice on the cake. It is easier to test if you know what to test exactly. Thanks to some good test coverage, a product feature matrix and the deep product knowledge of everyone involved in the testing process, we were sure not to miss out any feature of our application.

Conclusion

Thanks for reading this article. Major migrations in software engineering can be challenging especially in mature software projects. However, you should not be afraid to do such changes. Doing some research and planning before starting the migration helps a lot to mitigate the risk of failure. Let me know in the comments about your experiences.