Category Archives: Uncategorized

Deploying with Ember.js: a story

What is this all about?

This is the story of how we changed our Ember.js application deployment one night while no one was looking. This is not a strictly technical story, and the details may not matter; what is important is the journey we took and the steps you can follow to get your app deploying however you want.

Table of Contents

  1. What is this all about?
  2. Some Background (feel free to skip)
  3. Our Goals (the whole point of this)
  4. Our Journey
    1. Starts with a plan on a whiteboard
    2. Leads to Ember-cli-deploy
  5. The End
  6. Afterwords

Some Background (feel free to skip)

Our application – Ilios – is an installed solution and the code is in two parts. The API and management code is written in PHP and is installed by medical schools all over the world. Each installation is a little bit different, but they all require and consume our Ember.js frontend application. We don’t ship the frontend with the API codebase, it is provide separately and updated far more frequently. In the past we have shipped an index.html file with links to our AWS Cloudfront distribution containing the assets. This way the assets are stored in a CDN where they can be loaded quickly by users and easily kept up to date by our team without needing to understand each individual installation.

Our Goals (the whole point of this)

In order to take advantage of http/2 push and service workers we need to change the way the frontend is deployed. Our goal is to ship a single archive file containing the frontend assets including the app code, service workers, images, styles etc…

This archive must be tagged in such a way that we can have many versions in the wild at the same time with a default active version. Because the code for our API does change it must be possible to activate versions in such a way that v23 is the default frontend for API v1.1 and v34 is the default frontend for API v1.3.

Once the archive is downloaded and extracted by the API server it must be simple to parse and customize the files so that scripts and styles can be sent to the browser using http/2 push LINK headers and service workers can be placed at the root of our domain (where they must be do be effective).

Because we are constantly adding new features and fixing bugs we need to be able to release versions continuously.

TLDR

We want to ship a magical box to our customers and we want to ship it all the time.

Our Journey

Starts with a plan on a whiteboard

Distribute

  1. Build the app
  2. Archive the build artifacts
  3. Tag the archive with a version which combines both a unique ID for the code as well as the API version it works with
  4. Upload the archive to an S3 bucket
  5. Mark the most recent version so it is easy to find

Consume

  1. Download and extract the archive at
    • a) the most recent version
    • b) a specific version from the past
    • c) dev build we want to test
  2. Parse the the index file to extract configuration and assets
  3. Serve a modified index.html file to users

Leads to Ember-cli-deploy

This isn’t a surprise ember-cli-deploy has both the builtin tools and a plugin ecosystem to make deploying Ember.js apps very manageable. All we need to do is assemble a list of plugins that can meet our needs. As is the case with many ember addons, our journey really starts with Ember Observer where we find there is a whole category dedicated to plugins for ember-cli-deploy.

Some investigation and perusal of the deploy docs and a few minutes in the #ec-deploy channel in the Ember Slack Team will lead us to some standard choices and well supported solutions.

  • ember-cli-deploy is the foundation we can build on
  • ember-cli-deploy-build builds our app
  • ember-cli-deploy-revision-data can tag and activate versions
  • ember-cli-deploy-s3 uploads our assets and makes them public in an S3 bucket

A search for ‘archive’ in Ember Observer leads us to:

  • ember-cli-deploy-archive which takes our build assets and outputs a single tar archive

These combined knock out several of our needs.

Further investigation of ember-cli-deploy-s3 leads us to it’s companion ember-cli-deploy-s3-index which not only uploads to S3, it also manages versioning and activation.

And while looking around we also stumble upon ember-cli-deploy-json-config which conveniently parses our messy index.html file and outputs nice human and machine readable JSON that will be way easier for our API server to consume.

Looks like we don’t have to write any code at all! Just install and configure some plugins just the way we want and in the end just need to run a few instal commands:

ember install ember-cli-deploy
ember install ember-cli-deploy-archive
ember install ember-cli-deploy-build
ember install ember-cli-deploy-display-revisions
ember install ember-cli-deploy-json-config
ember install ember-cli-deploy-revision-data
ember install ember-cli-deploy-s3-index

Which helpfully creates a config/deploy.js file for us with some useful defaults:

module.exports = function(deployTarget) {
  var ENV = {
    build: {}
    // include other plugin configuration that applies to all deploy targets here
  };

  if (deployTarget === 'development') {
    ENV.build.environment = 'development';
    // configure other plugins for development deploy target here
  }

  if (deployTarget === 'staging') {
    ENV.build.environment = 'production';
    // configure other plugins for staging deploy target here
  }

  if (deployTarget === 'production') {
    ENV.build.environment = 'production';
    // configure other plugins for production deploy target here
  }

  // Note: if you need to build some configuration asynchronously, you can return
  // a promise that resolves with the ENV object instead of returning the
  // ENV object synchronously.
  return ENV;
};

We just need to configure our AWS info and we will something working. Note: I removed targets here as well as comments for brevity.

module.exports = function(deployTarget) {
  var ENV = {
    build: {}
    's3-index': {
      accessKeyId: '<our-access-key>',
      secretAccessKey: '<our-secret>',
      bucket: '<our-bucket-name>',
      region: '<our-bucket-region>'
    },
  };

  return ENV;
};

Next step is to try it out:

ember deploy development  --verbose
...
...
Pipeline complete

And when we got look in our S3 bucket low and behold there is an build.tar:HASH file in there! We’re really getting somewhere.

That version is nice, but remember we really need to know what API versions this build is compatible with. Oh, I see there is a prefix option we can use.

's3-index': {
  ...
  prefix: 'v1.22'
},

Deploy again and we now have a bucket with

build.tar:HASH (from our first deploy)
v1.22/
  build.tar:NEWHASH (from our second deploy)

That is exactly what the doctor ordered. Check that off our list and lets download that archive and see what it has. Hmmm… it has an index.html file, but no index.json file. Looking back at the --verbose output we can see why

+- didBuild
|  |
|  +- archive
- saving tarball of tmp/deploy-dist to tmp/deploy-archive/build.tar
- tarball ok
|  |
|  +- json-config
- generating `tmp/deploy-dist/index.json` from `tmp/deploy-dist/index.html`
- generated: `tmp/deploy-dist/index.json`
- added `index.json` to `context.distFiles`

The build.tar file is getting generated before the index.json file has been created. Since both of these plugins are doing their work inside the didBuild hook we just need to swap the order they run in. We can find out how to do that in the docs at http://ember-cli-deploy.com/docs/v1.0.x/configuration/#advanced-plugin-configuration

Looks like we just add a pipeline object to our deploy config like:

pipeline: {
  runOrder: {
    'archive': { after: 'json-config' },
  },
},

and… done! running ember deploy development again gives us exactly the build.tar we’re looking for.

The End

Seriously. That’s how the story ends. No brave journey into the darkness of code comments to figure out just how this is supposed to work. It just works. TM. It’s so simple we can just add it as a command at the end of our CI process and never think about it again.

Afterwords

I’m 100% head over heals in love with ember-cli-deploy. I find it incredibly freeing to be able to tinker with our deployment process to find just the right setup without needing to design and script each pice every time. The plugin ecosystem gives me the building blocks I can use to assemble our pipeline in whatever way makes sense so we can test new ideas and strategies fast and with very little friction.

I hope this was a useful overview of one deployment journey. Want to share your own? Tell me about a mistake I made or shower me with praise? Leave a comment here or find me at @iam_jrjohnson.

Testing in Ember.js, Part 2: ember-try and the Travis CI build matrix

Our Story So Far

The goal of automated testing is to find problems before your users do. Good tests do this by preventing bad code from being merged. A great continuous integration (CI) setup can catch problems in beta browsers and libraries in time to report them to their authors or fix your code before a release happens. By the end of this three part series you will have a great CI setup. Tests will automatically run against any browser you support and any future version of your dependencies.

In Part One, we covered using Sauce Labs and Travis CI to create your test matrix.

In Part Two, we will start testing our application against multiple versions of Ember using the excellent ember-try addon and create a Travis CI build matrix which will allow some options to fail without the entire test failing. The goal of these two improvements to our original setup is to see issues coming long before they become problems.

Getting Set Up

Part Two picks up right where we left off with a working ember-cli project and build configuration.

Let’s install the ember-try addon.

$ cd testing-sandbox
$ ember install ember-try

ember-try is a completely configurable way to test your code against upcoming versions of Ember. You may want to customize this later, but for now let’s add a file to testing-sandbox/config/ember-try.js with these contents:

/* jshint node: true */

module.exports = {
  scenarios: [
    {
      name: 'our-current',
      dependencies: {}
    },
    {
      name: 'ember-release',
      dependencies: {
        "ember": "ember#release"
      },
      resolutions: {
        "ember": "release"
      }
    },
    {
      name: 'ember-beta',
      dependencies: {
        "ember": "ember#beta"
      },
      resolutions: {
        "ember": "beta"
      }
    },
    {
      name: 'ember-canary',
      dependencies: {
        "ember": "ember#canary"
      },
      resolutions: {
        "ember": "canary"
      }
    }
  ]
};

That creates a few different build targets. The first one, our-current, is whatever version of Ember your app currently depends on, by leaving the dependencies blank it will use your current setup. The others are dynamically linked to the Ember release process for latest release, beta, and canary.

We can now run all of our tests against the beta version of Ember.js with a single command.

$ember try ember-beta test

Go ahead, give that a spin, it’s pretty great right?

You can also test everything in your config file with the command:

$ember try:testall

Automating with Travis CI

In Part One, we learned how to test against any browser and now we know how to test against any version of Ember.js. The only thing missing is a way to automate the entire process. We’re going to take advantage of Travis CI’s build matrix to organize our tests into discrete units.

We need to modify our Travis configuration to:

  1. Make a variable for the browser we are testing with
  2. Make a variable for the Ember.js version we are testing against
  3. Not start Sauce Connect unless we need it
  4. Combine all of this into a build matrix
  5. Allow some of these test combinations to fail

The new .travis.yml file looks like this:

---
language: node_js
node_js:
  - "0.12"

sudo: false

env:
  global:
    # Setup SauceLabs Credentials
    - SAUCE_USERNAME="YOUR_USER_NAME"
    - SAUCE_ACCESS_KEY="YOUR_ACCESS_KEY"
    # Some default values for our build matrix
    - START_SAUCE_CONNECT=false
    - EMBER_VERSION='our-current'
    - TESTEM_LAUNCHER='PhantomJS'

matrix:
  fast_finish: true
  allow_failures:
    - env: EMBER_VERSION='ember-beta'
    - env: EMBER_VERSION='ember-canary'
    - env: "TESTEM_LAUNCHER='SL_internet_explorer_11_Windows_8_1' START_SAUCE_CONNECT=true"
  include:
    - env: "TESTEM_LAUNCHER='SL_firefox_Windows_7' START_SAUCE_CONNECT=true"
    - env: "TESTEM_LAUNCHER='SL_internet_explorer_11_Windows_8_1' START_SAUCE_CONNECT=true"
    - env: "EMBER_VERSION='ember-beta'"
    - env: "EMBER_VERSION='ember-canary'"

cache:
  directories:
    - node_modules

before_install:
  - "npm config set spin false"
  - "npm install -g npm@^2"
 
install:
  - npm install -g bower
  - npm install
  - bower install

before_script:
  # Create a sauce tunnel only if we need it
  - if [ "$START_SAUCE_CONNECT" = true ]; then ember start-sauce-connect; fi

script:
  # run our tests against the Ember version and browser of our choice
  - ember try ${EMBER_VERSION} test --port=8080 --launch=${TESTEM_LAUNCHER} --skip-cleanup

after_script:
  # Destroy the sauce tunnel if we needed it
  - if [ "$START_SAUCE_CONNECT" = true ]; then ember stop-sauce-connect; fi

That’s it for Part Two! Tune in next week for a look at writing acceptance tests to take advantage of this setup.

If you have questions or see a mistake, tweet @iam_jrjohnson.

Updated:
5/6/15 – Changed the ember-try config to use a blank set of dependancies. Thanks @katiegengler for the suggestion.