At work we’ve recently finished upgrading the Rails version from 3.2 to 4.2 of our legacy application created in 2011.

I collected here the experience and the lessons I learned during the process along with some undocumented changes and solutions for them.

From 3.2 to 4.0

The work had been started before I joined the company so I decided to continue it with a few improvements.

Simplified workflow I followed

Listen to podcasts about how others did it

Go through the 4.0 Release notes

Go through the official upgrade guide

Go through the unofficial Fast Ruby upgrade guide

Make all tests green

Normal test on staging

Stress test on staging for a whole night

Deploy to one prod server and route more and more web traffic to it

Deploy to the remaining servers, let the queues and background jobs run for a while on Rails 4

Restore the web traffic

Organising the work

I defined 3 branches:

pre_rails4

rails4/master

post_rails4

Fortunately we could release half of the required changes before deploying Rails 4 itself. This gave us the very big advantage of releasing changes frequently in small chunks. Which came handy in debugging when something went wrong. We used the pre_rails4 branch for it.

A few changes had to wait until Rails 4 was stable, for these we used the post_rails4 branch.

The rest stayed on the rails4/master branch with frequent rebasing.

Lessons learned

Unfortunately this was highly neglected in the project before. So much that basically there was no upgrade in the previous one year and a half. As a result Rails 4 required a big pile of gems to be upgraded. Many were backward compatible, so we organised them into groups according to their risk and we deployed them in every Monday and Wednesday morning.

Waiting for management to allocate time officially

The first time I did everything by the Book. I worked on the upgrade only when I had a ticket in the actual sprint, which was a rather rare phenomenon despite all my pushing efforts. As a result a four-week task took us more than a year to finish.

Interesting contrast: after the major upgrade I went into a brief period of rebellion against the rules in the Book. Meaning, without any ticket or management approval I upgraded the patch version, fixed all the 10 broken tests, tested on staging and deployed the next Monday. It took 3-4 hours all together. And for my greatest surprise everyone was happy with it. As a consequence we fine-tuned the working process a bit to include cleanup and technical debt into the estimations and now both the engineering and business side is very happy!

Unexpected challenges

These minor changes were not advertised in any upgrade guide and yet, they rewarded me with some quality debugging time.

Rails 4 stopped truncating database string/text fields

More precisely as Moomaka pointed out in a Reddit comment, it was not Rails who truncated the content but MySQL itself. Rails 4 enabled strict mode in MySQL by default.

A broader description about the problem and a hotfix can be found here.

Ruby’s Logger is not extended by Rails 4

As a result the following code won’t set the datetime_format of the MyLogFormatter instance:

config . logger . formatter = MyLogFormatter . new config . logger . datetime_format = '...'

The solution:

config . log_formatter = MyLogFormatter . new config . log_formatter . datetime_format = '...'

Tip: How to validate if Rails overrides the datetime_format method?

Rails . logger . method ( :datetime_format ) . source_location = > [ "...activesupport/lib/active_support/core_ext/logger.rb" , 65 ] = > [ "...ruby/lib/logger.rb" , 285 ]

Database connection leaks in threads

The first stress test on staging provided a lot of ActiveRecord::ConnectionTimeoutError errors and the application stopped working completely.

It turned out that any database command running in a thread checks out a database connection from the pool and neglects to put it back. This is true for both Rails versions. The difference is Rails 3.2 calls the clear_stale_cached_connections! method when checking out a connection and there are no more available connections. Rails 4 doesn’t.

The solution was to call ActiveRecord::Base.clear_active_connections! manually or to use ActiveRecord::Base.with_connection with a block.

Thread . new do begin User . last ensure ActiveRecord : :Base . clear_active_connections ! end end

From 4.0 to 4.2

Leveraging all the experience from the previous upgrade this one went relatively smooth and fast. (Especially in the light of the rebelling spirit)

Was it worth it?

¡Totally!

One can be concerned about the cost of upgrading, but I’m quite confident the cost of not upgrading is even worst:

Security / bug issues (What would we tell our insurance company if we got hacked?)

Implementing and maintaining architecture / features exist in the next Rails versions

An old version is not really a selling point in the hiring process

…

On of my favourite new features is Active job. A unified API for background jobs. And finally we can send any sort of work to the background easily, especially sending emails.

From 4.2 to 5.0

It is also worth pointing out that Rails 6.0 RC1 has been released for quite some time, meaning Rails 6 is imminent. And therefore as soon as the first stable release is out of Rails 6, they will drop the support for 4.2.

Speaking of which, after dropping the support, there is nothing left except dropping the bass. (I’m sorry, the temptation pulled too strong to resist)

Our future plan is to try a bit different approach:

Instead of using a long-running branch we will follow how GitHub dealt with the problem. We will hook dual boot into our CI pipeline to require Rails 5 compatibility for new changes. And when everything is green, we will flip the switch.

Please let me know if you have met any unexpected challenge during an upgrade.