We use Behat for testing the Ilios code base. (We also use PHPUnit and Jasmine.) We started out using Cucumber but switched to Behat on the grounds that we should use PHP tools with our PHP project. Our thinking was that someone needing to dig in deep to write step code shouldn’t have to learn Ruby when the rest of the code was PHP.
We use Travis for continuous integration. Naturally, we needed to get our Behat tests running on Travis. Fortunately, there is already a great tutorial from about a year ago explaining how to do this.
Now let’s say you want to take things a step further. Let’s say you want your Behat tests to run on a variety of browsers and operating systems, not just whatever you can pull together on the Linux host running your Travis tests. One possibility is Sauce Labs, which is free for open source projects like Ilios.
Secure Environment Variables
Use the travis
Ruby gem to generate secure environment variable values for your .travis.yml
file containing your SAUCE_USERNAME
and your SAUCE_ACCESS_KEY
. See the heplful Travis documentation for more information.
Sauce Connect
You may be tempted to use the Travis addon for Sauce Connect. I don’t because, using the addon, Travis builds hang (and thus fail) when running the CI tests in a fork. This is because forks cannot read the secure environment variables generated in the previous step.
Instead, I check to see if SAUCE_USERNAME
is available and, if so, then I run Sauce Connect using the same online bash script (located in a GitHub gist) used by the addon provided by Travis. (By the way, you can check for TRAVIS_SECURE_ENV_VARS
if that feels better than checking for SAUCE_USERNAME
.)
The specific line in .travis.yml
that does this is:
- if [ "$SAUCE_USERNAME" ] ; then (curl -L https://gist.github.com/santiycr/5139565/raw/sauce_connect_setup.sh | bash); fi
Use the Source, Luke
Now it’s time to get Behat/Mink to play nicely with Sauce Labs.
The good news is that there is a saucelabs
configuration option. The bad news is that, as far as I can tell, it is not documented at the current time. So you may need to read the source code if you want to find out about configuration options or troubleshoot. Perhaps it’s intended to be released and documented in the next major release. Regardless, we’re using it and it’s working for us. Enable it in your behat.yml
file:
default:
extensions:
Behat\MinkExtension\Extension:
saucelabs: ~
Special Sauce
We keep our Behat profile for Sauce in it’s own file, because it’s special. Here’s our sauce.yml
file:
# Use this profile to run tests on Sauce against the Travis-CI instance
default:
context:
class: "FeatureContext"
extensions:
Behat\MinkExtension\Extension:
base_url: https://localhost
default_session: saucelabs
javascript_session: saucelabs
saucelabs:
browser: "firefox"
capabilities:
platform: "Windows 7"
version: 26
Note that we configured our app within Travis-CI to run over HTTPS. In a typical setting, you will want the protocol of your base_url
to specify HTTP instead.
Here’s the line in our .travis.yml
to run our Behat tests using the Sauce profile:
- if [ "$SAUCE_USERNAME" ] ; then (cd tests/behat && bin/behat -c sauce.yml); fi
Of course, if you’re using a different directory structure, you will need to adjust the command to reflect it.
That’s All, Folks!
I hope this has been helpful. It will no doubt be out of date within a few months, as things move quickly with Behat/Mink, Sauce Labs, and Travis-CI. I will try to keep it up to date and put a change log here at the bottom. Or if a better resource for this information pops up, I’ll just put a link at the top. Thank you for reading!