Ricardo Vila, software engineer at TEIMAS, gives a first-hand look at his work developing Teixo, a Saas product that was originally written in Rails 2.3.
.jpg)
Ricardo Vila, ingeniero de software en TEIMAS, cuenta de primera mano su trabajo desarrollando Teixo, un producto Saas que fue escrito originalmente en Rails 2.3.
En 2021 comenzamos un viaje para actualizar Teixo a la última versión de Rails, y estos posts son mi diario:
Este artículo está únicamente disponible en inglés.
We are TEIMAS, a leading software company specializing in digitalizing the waste value chain, one of the few worldwide. We assist companies in transitioning towards a circular economy and decarbonization to safeguard resources, the environment, and public health. Our software solutions promote the reintroduction of waste into the production chain, reducing the need for new resource consumption.
One of our flagship solutions is Teixo, a SaaS software specifically designed for Waste Management Companies. Teixo offers a wide range of features, including:
Teixo is the leading waste management tool in Spain, implemented in over 700 waste management facilities with more than 3000 active daily users.
This series of posts focuses on the project of updating our flagship product, Teixo, without any downtime and ensuring uninterrupted service to all of our customers.
This project involved not only technical staff but also members of our customer success department and, of course, many users and clients. Therefore, these posts will cover not only technical issues but also customer management and procedures to ensure the quality of Teixo.
Teixo's first lines of code were written in 2008, using Rails 2.3 over Ruby 1.8.3, which was the leading technology of the time. Teixo uses a MySQL database, which despite its large size, has excellent performance.
Over the last twelve years, Teixo has evolved at a rapid pace, adding features and adapting to new infrastructure needs with great success. The main components of Teixo include:
However, despite updating Ruby, Teixo faced many problems, such as outdated gems, an obsolete Activerecord API, deployment troubleshooting, etc. This made new developments more complex and error-prone over time.
In 2021, we decided to reduce our technical debt and scheduled a project to ensure the viability of Teixo for at least 10 more years. We decided to update Teixo's Rails version in different steps or phases, one for each major version of Rails:
At this point, we still weren't sure whether it would be better to split these steps into smaller ones (one for each minor version, i.e., Rails 3.0, later Rails 3.1, and finally 3.2), or do each major change at a time. Although Rails has good documentation on updating apps from one version to the next one, the Rails Guides, this project was still extremely complex.
To help understand the complexity of the project, here is some data about Teixo:
Declared Gems 59, Total Gems 134.
By the end of 2021, we began to define "The Plan," which had several key requirements:
We also began searching for companies that could assist us with this process - a "Guide" for our journey. We found a Philadelphia-based company specializing in updating Rails projects and started working with them in the early phase of checking over Teixo's codebase to prepare a report about the steps and potential problems we might encounter in this project.
After completing the analysis, they proposed a step-by-step update process, one for each minor Rails version (i.e., Rails 3.0, 3.1, 3.2, 4.0, and so on). However, we decided that this approach would take too long to complete, so we opted for larger steps to achieve the update in a reasonable timeframe. Fortunately, we had a strong QA and Customer Success department that helped us test and validate the update steps quickly and reliably, even with significant platform changes.
We knew, and the analysis confirmed, that we needed to prepare for this update process. For several months, we enforced QA on Teixo, developing many new automatic tests on our CI environment and expanding the existing ones. We also strengthened our Customer Success department protocols for checking Teixo's quality.
The analysis also proposed a dual-boot mechanism, having one codebase suitable to boot on two versions of Rails. However, we thought that this approach would be very challenging to handle and would increase the overhead of an already very complex project. Instead, we decided to tackle the project in a more streamlined way, making the necessary code changes on each step of the project.
The report included a summary of the changes needed to be made in Teixo for each Rails step, which was an adapted version of the Rails Guides for updating.
We propose to handle the project by simultaneously developing two versions of Teixo at each step of the process. For example, during the first phase, one version will use Rails 2.3, while the other will use Rails 3.2.
In the staging and production environments, both versions should work with the same database schema. Only one version should handle database migrations, but both versions should be compatible with the schema. This requires particular care with migrations, ensuring that both versions of the code have the same migrations, or at the very least, do not delete attributes/columns in only one version.
We needed to incorporate the update process into our regular development workflow, so we made some adjustments to our development process.
At each step of the update process, there are two phases of work to be completed.
If we had chosen to take smaller steps (minor Rails version changes), the overhead of the second phase would have been excessive, particularly for the Customer Success department.
Meanwhile, the normal Teixo development workflow continued. During this second phase, we had to change our usual procedure in the following aspects:
We also modified our process for new Teixo releases to handle this duality.
In April 2022, we began working on the first and by far the most complex step we would take: updating from Rails 2.3 to 3.2 without any intermediate steps.
We hired one expert developer from our 'Guide' company to team up with two senior developers from TEIMAS. Unfortunately, this collaboration did not improve the project as expected, due to several obstacles that made collaboration difficult:
Ultimately, our collaboration ended on friendly and mutual terms before completing the first step. As a result, we finished the first update without external help.
Before starting work with a new branch for Rails 3.2, we put in a significant effort to increase Teixo's automated test coverage, reaching almost 70%. This work was one of the most critical components for the project's success.
Additionally, we addressed Deprecation Warnings, such as 'ActiveRecord::Base#class_name is deprecated,' and implemented other improvements such as organizing the code more effectively and cleaning up unused parts of the project.
This section covers the various technical changes, problems, and issues we encountered when migrating from Rails 2.3 to 3.2. This work was mostly carried out in the first part of the update step, but some was also done during the 'reactive work' phase to address previously undetected issues and errors that were brought to our attention by the Customer Success department and even some trusted customers.
As previously mentioned, we upgraded to a Rails 3.2 LTS version supported by Makandra, a team of veteran Rails developers and operations engineers, as they define themselves.
We use Bundler and once we’d updated the gemfile to use:
gem 'rails', '~> 3.2.22.27'
we had to fix some Gem issues.
1. Our version of the Mysql2 gem (0.5.3) was incompatible with this version of Rails. As a result, we had to switch to a different version of the Mysql2 gem, which was available from the Makandra repository:
gem 'mysql2', git: 'https://github.com/makandra/mysql2', branch: 'master'
2. We also had to fork Makandra's activerecord gem to modify it to support Mysql gems greater than ours. This involved simply changing one line on lib/active_record/connection_adapters/mysql2_adapter.rb.
require 'active_record/connection_adapters/abstract_mysql_adapter'
gem 'mysql2', '> 0.3.10'
require 'mysql2'
module ActiveRecord
class Base
....
Configuration and several other files required dependencies using the File API, which had changed. Consequently, we had to make changes in several areas, including:
require File.dirname(__FILE__) + '/../config/boot'
To
require File.expand_path('../../config/boot', __FILE__)
We also had to update several boot and configuration files along with many initializers.
Config: confg.ru for example, was changed on various lines, mainly changed:
run ActionController::Dispatcher.new
to
run Teixo::Application
Application: We had to build a new config/application.rb with the app configuration (extracted from the config/environment.rb file) declaring Teixo app as:
module Teixo
class Application < Rails::Application
Boot: config/boot.rb was simplified to simply load the Gemfile and bundler.
require 'rubygems'
# Set up gems listed in the Gemfile.
ENV['BUNDLE_GEMFILE'] ||= File.expand_path('../../Gemfile', __FILE__)
require 'bundler/setup' if File.exists?(ENV['BUNDLE_GEMFILE'])
Rakefile: has to define how to load configuration and the Application
require File.expand_path('../config/application', __FILE__)
...
Teixo::Application.load_tasks
Routes: The most time-consuming change was rewriting the config/routes.rb file. Route declaration changed significantly from Rails 2 to 3. Despite having a tool available to translate routes from 2 to 3, our routes.rb file was not very well defined. As a result, we had to manually convert many of the routes to the new format.
It was very handy to try routes helpers from the console (when we got it to work), for example:
irb(main):002:0> app.admin_users_path
=> "/admin/users"
Despite of the Gems update and configuration changes the project was not able to launch a Rails console nor the Server. We needed to make several more changes.
RAILS_ENV was no longer supported, we had to change it to Rails.env in all the project.
All references to the package ActionController::xxxxxx had changed to ActionDispatch::xxxxx (for example ActionController::Routing::Routes)
All scope declarations must change from ‘named_scope’ to simply ‘scope’. Also, some scopes on superclasses didn’t work well in subclasses so we had to rewrite some of them using ‘scoped’. For example:
named_scope :not_draft, lambda { {:conditions => ["#{table_name}.state != ?", DOCUMENT_STATES[:draft]]} }
Became:
def self.not_draft
scoped(:conditions => ["#{table_name}.state != ?", DOCUMENT_STATES[:draft]])
end
We changed a few attr_accessor_with_default used in Teixo because it was no longer supported. We fixed each case depending on the need. Mostly using attr_writer to declare the field and/or defining a getter method.
Saving without validations was no longer available with a ‘save(false)’ call, now it is save(:validate => false).
Several helper methods used in Haml (and Erb) partials no longer worked using the - operator, we had to change it to the = operator.
- form_tag session_path, :id => 'login_form' do
Changed to
= form_tag session_path, :id => 'login_form' do
The flash method on Rails 2 returned an object that responded to the Hash API, on Rails 3 this is no longer true so we had to call to_h on every use:
if flash.values.empty?
Changed to
f_hash = flash.to_h
if f_hash.values.empty?
The use of @template inside controllers is no longer supported. Instead, it is recommended to use the view_context method (In Rails 3, the new AbstractController was introduced).
.png)
