We’re stuck in the past – or parts of our code base is

DataFox has been using Ember, a powerful Javascript frontend framework, for years, and over these years it has continuously advanced. Unfortunately we had not been benefiting from those advancements as we were using outdated version of Ember: as of beginning of 2018 we were on Ember 2.2, while the latest long term support was 2.18. Worse, we were not using Ember-CLI, a command line interface for running and building an Ember based app. Instead we used Grunt to build and serve our app, which was a common pattern for a lot of Ember applications before Ember-CLI was released.

However, we’re now on the road forward with a switch over to using Ember-CLI and a plan to upgrade to the LTS release of Ember. This blog post will go over what options we considered as we determined the next step for our Ember application and our process of achieving that goal.

We considered the options

To give you some context into why we made our decision, we should tell you a little bit about us: DataFox is a startup with a small engineering team of about 10. Of those 10, around 4 engineers are full stack and are often working with our user-facing Ember application. As you can imagine with a startup, we are constantly balancing working in tech debt related issues versus working on updating or creating features. Ember was the frontend framework chosen when the company was founded over 4 years ago, so we have seen the framework adapt and change over time, and grow to over 200 components and nearly 100 controllers. However, this also means that there does exist some amount of tech debt in our code.

We had a lot of options to consider before pursuing this path:

Option 1: Upgrade Ember LTS without switching to ember-cli

We decided not to go down this path as there’s a lack of documentation and support for this path.

Option 2: Abandon Ember and use React or another similar framework

React is an increasingly popular framework that does have its advantages, but it was also prohibitively time consuming, we don’t have much in-house expertise, and we like Ember – it’s a solid choice for a frontend framework. In addition, our internal tools also used Ember, and with such a small engineering team it did not seem like the right time to introduce another stack to our system.

Option 3: Recreate the app on newest version of Ember/Ember-CLI

This would also have been prohibitively time consuming due to the size and functionality of our app.

The option we went with: Convert to Ember-CLI and continue the upgrade process after.

We decided to go down this path due to all of the advantages with it: It’ll make upgrading to LTS easier, which will allow us to take advantage of newer functionality and security fixes. It’ll allow us to use ember add-ons, which are libraries that plug into ember to add new functionality/components. I went to Embercon and “there’s an add-on for that” was a common phrase, it’s a core part of most Ember usage! Add-ons will allow us to save development time as we no longer have to build common components ourselves and instead use community-tested elements. Also, since testing is baked into ember-cli, we’ll be able to test our front-end more easily (vs. currently, we only have a couple ghost inspector tests running on production and staging). Additionally, some of our smaller apps use Ember-CLI already so we’ll be more consistent, which will make all the apps easier to maintain. In general, we believe upgrading to Ember-CLI will give us a much better development experience – debugging, documentation, support, etc.

We again considered the options

Once we decided to do the upgrade, we had to decide how, so we explored multiple methods: Manually upgrade was quickly vetoed. Ember-CLI expects files to be in a different format and that’d have been over one thousand files to manually update.

Serve our existing files from the assets directory and manually convert them as we use them. This was also fairly quickly vetoed as we’d still be manually converting a lot of files, though this has advantage of not having to stop development to do the conversion, and would have likely been the quickest solution but with the largest overhead/tech debt that we were not willing to take on. What we went with: create an automated conversion script. This allowed us to not to stop development as we develop the script or test the changes, and limited tech debt after the conversion is done!

We scripted it

Now that we had the path forward, we needed to make that script. Thankfully a lot of work had already been done via the ember-cli-migrator project . We installed 2.2 version of ember-cli and used it to create a fresh install of an Ember 2.2 app, then used the migrator to convert our files over. However, it only got us part of the way there.

We tackles those issues via multiple directions:

Created the migrator script:
  • Used ember-cli-migrator via wrapper script so we could add shell commands to handle some of the conversion, which were often faster and easier to write.
  • The migrator had a few bugs (https://github.com/fivetanley/ember-cli-migrator/pull/83 https://github.com/fivetanley/ember-cli-migrator/pull/84), like some dependant libraries being outdated and the script being unable to handle es6 code, so we patched those issues and submitted pull request to the repo.
  • Updated the migrator to convert using a folder structure instead of flat directories – e.g. the migrator puts all controllers one level deep under controllers, whereas we want, for example, ‘account-scoring-draft.js” file instead be converted to “account-scoring/draft.js” (which matches our existing template structure). Being that we wanted some folders to contain hyphens, e.g. account-scoring, we ended up doing some specific logic to handle that and did not contribute this change back to the main ember-cli-migrator repository. We also had to create a custom Ember resolver to match how we wanted it to resolve old references with the new directory structure.
  • Created some empty files that used to have require statements we needed for grunt – some hard coded rm’s in the wrapper script fixed this.
  • It didn’t migrate initializers correctly. I looked a bit into how to implement this generically in ember-cli-migrator, but when considering the time it’d take me to learn how to use recast (the library migrator used to convert the code) in that way, and considering for how few initializers we have, and that the initializers are rarely updated, I decided to instead manually convert them then copy them over after the migrator via the script.
Updating our code base
  • We updated our original code base to fix some situations the migrator could not. These we commited to master (after code review) and deployed like other updates to our app while we worked on the migration.
  • In our original code base we’d use the App name space in unusual ways (for example we had an error handling function App.onErrror()) , which did not convert correctly. Instead we reformatted those to use proper Ember workflows – for example, the error handler is now part of an error handling service.
  • Migrator did not know how to associate variables outside the object definition with the object that used it (e.g. 1 file with 2+ controllers and const someVariable outside the definition of those controllers). Since this was only a few files, we split the files up to ease the conversion.
Updated the Ember-CLI App
  • Ember-CLI defaults to jshint instead of our preferred eslint, so we added the add-on ember-cli-eslint (and ran eslint –fix against the migrated code at the end of the script).
  • Our dependencies had been manually added to a dependencies folder. Now that we have ember-cli, we converted them to use a combination of npm/bower when possible, and the vendor directory if neither worked, then updated ember-cli-build.js to include them from npm/bower/vendor. Later version of ember have deprecated bower, but not 2.2 and some libraries we did not find the versions we needed via NPM.

Testing 1 2 3

With those changes, we got the conversion working, but we needed a way to test it. We updated our express route to serve the ember-cli generated code when a per-user flag was enabled, and sent it off to our QA to test. The benefit of this was we could continue testing the Ember-CLI version of our app while development on features continued – the only caveat was that we had to run the migrator script daily before our deploy. Once we fixed the bugs we found, we started enabling it for our internal team for testing.

We’ve now switched over to Ember-CLI completely and are excited to be moving forward!