Testing in Ember.js, Part 3: mock data with ember-cli-mirage

When Last We Left Our Heroes…

The goal of automated testing is to find problems before your users do. Good tests do this by preventing bad code from being merged. A great continuous integration (CI) setup can catch problems in beta browsers and libraries in time to report them to their authors or fix your code before a release happens. By the end of this three part series you will have a great CI setup. Tests will automatically run against any browser you support and any future version of your dependencies.

In Part One, we covered using Sauce Labs and Travis CI to create your test matrix.

In Part Two, we covered testing our application against many versions of Ember.

In Part Three, we will write some acceptance tests and use the awesome ember-cli-mirage addon to provide controlled mock data to our tests and development environment.

Getting Set Up

Part Three picks up right where we left off with a working ember-cli project and build configuration. This is not a tutorial on Test Driven Development so we’re going to start out with a working example and then test it.

First setup our ember-data models:

$ cd testing-sandbox
$ ember g model fruit title:string color:belongsTo
$ cat app/models/fruit.js
import DS from 'ember-data';
export default DS.Model.extend({
  title: DS.attr('string'),
  color: DS.belongsTo('color', {async: true})
});

$ ember g model color title:string fruits:hasMany
$ cat app/models/color.js 
import DS from 'ember-data';

export default DS.Model.extend({
  title: DS.attr('string'),
  fruits: DS.hasMany('fruit', {async: true})
});

Then a simple route:

$ ember g route colors --path='/colors'

Modify app/routes/colors.js to get all the colors:

import Ember from 'ember';

export default Ember.Route.extend({
    model(){
      return this.store.find('color');
    }
});

Setup a template to list the colors and their fruits app/templates/colors.hbs

<ul>
  {{#each model as |color|}}
    <li>{{color.title}}
      <ul>
        {{#each color.fruits as |fruit|}}
          <li>{{fruit.title}}</li>
        {{/each}}
      </ul>
    </li>
  {{/each}}
</ul>

Setup ember-cli-mirage

Let’s install the ember-cli-mirage addon.

$ ember install ember-cli-mirage

Now we need to configure our basic API routes. Mirage creates a basic configuration file for you at app/mirage/config.js. We just need to add a few lines for our new models:

export default function() {
  this.get('/colors');
  this.get('/colors/:id');
  this.get('/fruits');
  this.get('/fruits/:id');
};

For each of our models we need a factory so our tests can create new data. As ember-cli-mirage matures, generators for your factories will be added. For now, you have to make them on your own. We need one for fruit and one for color.

Create the file app/mirage/factories/fruit.js:

import Mirage from 'ember-cli-mirage';

export default Mirage.Factory.extend({
  title: (i) => `fruit ${i}`,
  color: null
});

…and the file app/mirage/factories/color.js:

import Mirage from 'ember-cli-mirage';

export default Mirage.Factory.extend({
  title: (i) => `color ${i}`,
  fruits: []
});

Wow. Thats was some serious setup; take heart that we’re done now and we can finally write a test.

Finally a test!

$ember g acceptance-test colors

Add some fixture data and a test to your new file at tests/acceptance/colors-test.js:

test('visiting /colors', function(assert) {
  //turn on logging so we can see what mirage is doing
  server.logging = true;
  
  //make our first color, its gets an id of one
  server.create('color', {
    //fill this color with some fruits (they don't exist yet, thats next)
    fruits: [1,2,3,4,5]
  });
  //now lets create a bunch of fruits and link them to our color
  server.createList('fruit', 5, {
    color: 1
  });
  //want another color? - just add it.
  server.create('color', {
    fruits: [6,7]
  });
  server.createList('fruit', 2, {
    color: 2
  });
  visit('/colors');

  andThen(function() {
    assert.equal(currentURL(), '/colors');
    //this is a stupid test, but hey its a tutorial, what did you expect?
    assert.equal(find('li').length, 9);
  });
});

Yup, I wrote that test for you. This isn’t a lesson on Test Driven Development. If you want that watch “Test Driven Development By Example”. The important part here is that we create fresh testing data with every test using server from ember-cli-mirage. You can be in complete control of what is passed to your application so you can check for any condition.

Final Thoughts

We’re just about out of time and we covered a lot. You still have some test writing to do and I wish there was an addon to do that for you. Until then you can take solace in the knowledge that your testing infrastructure is a foundation you can build your reputation on.

Until next time, Internet friends: If you liked it or hated it let me know @iam_jrjohnson.

Testing in Ember.js, Part 2: ember-try and the Travis CI build matrix

Our Story So Far

The goal of automated testing is to find problems before your users do. Good tests do this by preventing bad code from being merged. A great continuous integration (CI) setup can catch problems in beta browsers and libraries in time to report them to their authors or fix your code before a release happens. By the end of this three part series you will have a great CI setup. Tests will automatically run against any browser you support and any future version of your dependencies.

In Part One, we covered using Sauce Labs and Travis CI to create your test matrix.

In Part Two, we will start testing our application against multiple versions of Ember using the excellent ember-try addon and create a Travis CI build matrix which will allow some options to fail without the entire test failing. The goal of these two improvements to our original setup is to see issues coming long before they become problems.

Getting Set Up

Part Two picks up right where we left off with a working ember-cli project and build configuration.

Let’s install the ember-try addon.

$ cd testing-sandbox
$ ember install ember-try

ember-try is a completely configurable way to test your code against upcoming versions of Ember. You may want to customize this later, but for now let’s add a file to testing-sandbox/config/ember-try.js with these contents:

/* jshint node: true */

module.exports = {
  scenarios: [
    {
      name: 'our-current',
      dependencies: {}
    },
    {
      name: 'ember-release',
      dependencies: {
        "ember": "ember#release"
      },
      resolutions: {
        "ember": "release"
      }
    },
    {
      name: 'ember-beta',
      dependencies: {
        "ember": "ember#beta"
      },
      resolutions: {
        "ember": "beta"
      }
    },
    {
      name: 'ember-canary',
      dependencies: {
        "ember": "ember#canary"
      },
      resolutions: {
        "ember": "canary"
      }
    }
  ]
};

That creates a few different build targets. The first one, our-current, is whatever version of Ember your app currently depends on, by leaving the dependencies blank it will use your current setup. The others are dynamically linked to the Ember release process for latest release, beta, and canary.

We can now run all of our tests against the beta version of Ember.js with a single command.

$ember try ember-beta test

Go ahead, give that a spin, it’s pretty great right?

You can also test everything in your config file with the command:

$ember try:testall

Automating with Travis CI

In Part One, we learned how to test against any browser and now we know how to test against any version of Ember.js. The only thing missing is a way to automate the entire process. We’re going to take advantage of Travis CI’s build matrix to organize our tests into discrete units.

We need to modify our Travis configuration to:

  1. Make a variable for the browser we are testing with
  2. Make a variable for the Ember.js version we are testing against
  3. Not start Sauce Connect unless we need it
  4. Combine all of this into a build matrix
  5. Allow some of these test combinations to fail

The new .travis.yml file looks like this:

---
language: node_js
node_js:
  - "0.12"

sudo: false

env:
  global:
    # Setup SauceLabs Credentials
    - SAUCE_USERNAME="YOUR_USER_NAME"
    - SAUCE_ACCESS_KEY="YOUR_ACCESS_KEY"
    # Some default values for our build matrix
    - START_SAUCE_CONNECT=false
    - EMBER_VERSION='our-current'
    - TESTEM_LAUNCHER='PhantomJS'

matrix:
  fast_finish: true
  allow_failures:
    - env: EMBER_VERSION='ember-beta'
    - env: EMBER_VERSION='ember-canary'
    - env: "TESTEM_LAUNCHER='SL_internet_explorer_11_Windows_8_1' START_SAUCE_CONNECT=true"
  include:
    - env: "TESTEM_LAUNCHER='SL_firefox_Windows_7' START_SAUCE_CONNECT=true"
    - env: "TESTEM_LAUNCHER='SL_internet_explorer_11_Windows_8_1' START_SAUCE_CONNECT=true"
    - env: "EMBER_VERSION='ember-beta'"
    - env: "EMBER_VERSION='ember-canary'"

cache:
  directories:
    - node_modules

before_install:
  - "npm config set spin false"
  - "npm install -g npm@^2"
 
install:
  - npm install -g bower
  - npm install
  - bower install

before_script:
  # Create a sauce tunnel only if we need it
  - if [ "$START_SAUCE_CONNECT" = true ]; then ember start-sauce-connect; fi

script:
  # run our tests against the Ember version and browser of our choice
  - ember try ${EMBER_VERSION} test --port=8080 --launch=${TESTEM_LAUNCHER} --skip-cleanup

after_script:
  # Destroy the sauce tunnel if we needed it
  - if [ "$START_SAUCE_CONNECT" = true ]; then ember stop-sauce-connect; fi

That’s it for Part Two! Tune in next week for a look at writing acceptance tests to take advantage of this setup.

If you have questions or see a mistake, tweet @iam_jrjohnson.

Updated:
5/6/15 – Changed the ember-try config to use a blank set of dependancies. Thanks @katiegengler for the suggestion.

Testing in Ember.js, Part 1

[Update] 10/28/17 – Thanks to @DanstonEvans corrected the browser string for firefox on SauceLabs.

The Big Picture

The goal of automated testing is to find problems before your users do. Good tests do this by preventing bad code from being merged. A great continuous integration (CI) setup can catch problems in beta browsers and libraries in time to report them to their authors or fix your code before a release happens. By the end of this three part series you will have a great CI setup. Tests will automatically run against any browser you support and any future version of your dependencies.

Requirements for this guide are Ember.js > 1.10 and Ember CLI > 0.2.3. It may be entirely possible to do this without Ember CLI, but I wouldn’t know how.

In Part One, we will cover using Sauce Labs and Travis CI to create your test matrix.

Getting Set Up

If you’ve never used Ember CLI before, you should follow their instructions to install all dependencies.

Now let’s create a new sandbox to play in:

$ ember new testing-sandbox
$ cd testing-sandbox
$ ember test --server

Congrats! You now have a brand new Ember.js app and running tests in both PhantomJS and Chrome. Go ahead and leave that console window open and create a new one. Tests will keep running in the original window and track all the changes we make.

$ cd testing-sandbox
$ ember g acceptance-test welcome-page

Your test console should now record a failure indicating:

✘ UnrecognizedURLError: /welcome-page

Open testing-sandbox/tests/acceptance/welcome-page-test.js in your favorite editor and make it look like this:

import Ember from 'ember';
import {
  module,
  test
} from 'qunit';
import startApp from 'testing-sandbox/tests/helpers/start-app';

var application;

module('Acceptance: WelcomePage', {
  beforeEach: function() {
    application = startApp();
  },

  afterEach: function() {
    Ember.run(application, 'destroy');
  }
});

test('we should be welcoming', function(assert) {
  visit('/');

  andThen(function() {
    assert.equal(currentURL(), '/');
    var title = find('#title');
    assert.equal(title.text(), 'Welcome to Ember.js');
  });
});

Save that and all of your tests should pass. We are ready to get started with multi-browser testing.

Test Multiple Browsers in the Cloud

Sauce Labs is a service for running your tests against a huge variety of browsers. We’re going to abstract a lot of the complexity of using Sauce Labs by taking advantage of the excellent ember-cli-sauce addon. First, you will need Sauce Labs credentials. You can start a free trial or, if your project is open source, you can sign up for Open Sauce. When you are done, take note of your user name and access key. You will need them later.

Let’s install the addon:

$ember install ember-cli-sauce

Now we can add additional browsers to our testem.json file. Testem calls these launchers:

  $ ember sauce --browser='firefox' --platform='Windows 7'
  $ ember sauce --browser='internet explorer' --version=11 --platform='Windows 8.1'

Lets run some tests!

First we have to export our sauce credentials as environment variables.

$export SAUCE_USERNAME="YOUR_USERNAME"
$export SAUCE_ACCESS_KEY="YOUR_ACCESS_KEY"

Then we fire up a proxy tunnel so Sauce Labs browsers can get to our local Ember.js server.

$ember start-sauce-connect

Then we launch the actual tests.

$ember test --launch='SL_firefox_public_Windows_7,SL_internet_explorer_11_Windows_8_1'

You should see something like:

ok 1 Firefox 37.0 - Acceptance: WelcomePage: we should be welcoming
ok 2 Firefox 37.0 - JSHint - acceptance: acceptance/welcome-page-test.js should pass jshint
ok 3 Firefox 37.0 - JSHint - .: app.js should pass jshint
ok 4 Firefox 37.0 - JSHint - helpers: helpers/resolver.js should pass jshint
ok 5 Firefox 37.0 - JSHint - helpers: helpers/start-app.js should pass jshint
ok 6 Firefox 37.0 - JSHint - .: router.js should pass jshint
ok 7 Firefox 37.0 - JSHint - .: test-helper.js should pass jshint
ok 8 IE 11.0 - Acceptance: WelcomePage: we should be welcoming
ok 9 IE 11.0 - JSHint - acceptance: acceptance/welcome-page-test.js should pass jshint
ok 10 IE 11.0 - JSHint - .: app.js should pass jshint
ok 11 IE 11.0 - JSHint - helpers: helpers/resolver.js should pass jshint
ok 12 IE 11.0 - JSHint - helpers: helpers/start-app.js should pass jshint
ok 13 IE 11.0 - JSHint - .: router.js should pass jshint
ok 14 IE 11.0 - JSHint - .: test-helper.js should pass jshint

1..14
# tests 14
# pass  14
# fail  0

# ok

Wasn’t that awesome? You just tested your code in two browsers. You can add anything you want to testem.json. Go nuts!

When you are done testing remember to kill the tunnel we opened.

$ember stop-sauce-connect

Making It Automatic with Travis CI

The last piece of this puzzle is to use Travis CI to run these tests for you every time you commit code. Update your .travis.yml file to run Sauce Labs tests. You will need to tell Travis CI what your Sauce Labs credentials are in the env section:

---
language: node_js
node_js:
  - "0.12"

sudo: false

env:
  global:
    #set these here becuase they get pulled out by testem saucie
    - SAUCE_USERNAME="YOUR_USER_NAME"
    - SAUCE_ACCESS_KEY="YOUR_ACCESS_KEY"

cache:
  directories:
    - node_modules

before_install:
  - "npm config set spin false"
  - "npm install -g npm@^2"

install:
  - npm install -g bower
  - npm install
  - bower install

before_script:
  # Create a sauce tunnel
  - ember start-sauce-connect

script:
  - ember test --launch='SL_firefox_Windows_7,SL_internet_explorer_11_Windows_8_1' --port=8080

after_script:
  # Destroy the sauce tunnel
  - ember stop-sauce-connect

You are well on your way to being a cross-browser testing hero! In my next post I will take you through using the ember-try addon to test your code against upcoming Ember.js versions.

If you have questions or see a mistake, you can use the comments here or tweet @iam_jrjohnson.

Ember CLI, Heroku, and You

Warning, this is old information and way more work than you need to do. The solution we are using now is the excellent heroku-buildpack-ember-cli.

A disclaimer: This is not for use in production. Doing this for a production app would be a bad decision.

The problem: Developers on the Ilios Project need to be able to share their changes with other team members. While it is possible to deploy a static Ember CLI app nearly anywhere, we want to include our mock API so everyone is looking at the same data.

The solution: Use Heroku to host an Ember CLI application running its built in Express server.

Continue reading Ember CLI, Heroku, and You

Introducing Amalgamatic

“Search!” by Jeffrey Beall licensed CC BY-ND 2.0

Academic libraries offer many resources, but users cannot be expected to search in, say, a half dozen different interfaces to find what they’re looking for. So academic libraries typically offer federated search.

Sometimes, a solution is purchased. Many libraries, for example, use 360 Search.

Here at UCSF, we are among the libraries that have built our own federated search. Twice.

Continue reading Introducing Amalgamatic

Random Forests and Datashare at the CDL Code Jam

The California Digital Library hosted a code jam earlier this month at the Oakland City Conference center.

This gathering brought together librarians and developers from University of California campuses for a series of working meetings with an eye toward system-wide projects, especially involving data curation.

In the spirit of an informal, code-jam style meeting, I presented a bit on my recent experiments using machine learning to categorize data. As a starting example, I applied a random forest to suggest subject keywords for data sets uploaded to the recently launched DataShare website.

Continue reading Random Forests and Datashare at the CDL Code Jam

Redesign Your Website in 4,000 Easy Steps

Earlier this month, Rich Trott and I delivered a session at the University of California Computing Services Conference (UCCSC) in San Francisco. It was about our experience using an approach of continuous iterative improvements and frequent feedback to help keep our site fresh and meeting user needs. We talked about why this approach has been working better than the tradition complete redesign that might happen every few years (or not.)

If you can’t wait to hear more, see the slides with notes.

Or if you’re more of the video type, you can check that out too.

Let us know about your experiences using this kind of approach to website upkeep and positive user experience. What works for your site or organization?

Photo by chexee