All posts by Rich Trott

Improving Summon/EZproxy/Shibboleth Login Experience With Custom JavaScript

If you use Summon’s custom auth banner/VPN banner with Shibboleth and EZproxy, you may have noticed an unfortunate side effect. If you do a search, and then click the banner to log in, you will be redirected to the start page rather than back to your search results.

Fixing this means tapping into their app with custom JavaScript. The front end of Summon is built on AngularJS 1.2.27 and ships with jQuery.  Here’s how we’re currently doing it:

(function () {
  var waitingForInitialLoad = true;

  var fixBannerLink = function () {
    var links = $('.vpnBanner.customAuthBanner div a');
    if (waitingForInitialLoad && links.length === 0) {
      setTimeout(fixBannerLink, 100);
      return;
    } else {
      waitingForInitialLoad = false;
      window.addEventListener('hashchange', fixBannerLink);
    }
    
    links.each(function() {
      if (/^Off the UCSF network/.test(this.text)) {
        $(this).attr('ng-href', 'https://ucsf.idm.oclc.org/login?qurl=' + encodeURIComponent(location.href));
        $(this).attr('href', 'https://ucsf.idm.oclc.org/login?qurl=' + encodeURIComponent(location.href));
      }
    });
  }

  $().ready(fixBannerLink);
}());

Most embarrassing hack: The /^Off the UCSF network/ regular expression is of course matching the text of the link. You’ll need to update that to match the text in your own link. (Or find a less embarrassing hack and submit it as a pull request in the GitHub repository.)

Also, our EZproxy URL prefix is hard-coded in there a few times. Change to your own, and make sure you use the qurl URL parameter rather than url.

If you have any questions, feel free to open an issue in the GitHub repository.

Introducing Amalgamatic

“Search!” by Jeffrey Beall licensed CC BY-ND 2.0

Academic libraries offer many resources, but users cannot be expected to search in, say, a half dozen different interfaces to find what they’re looking for. So academic libraries typically offer federated search.

Sometimes, a solution is purchased. Many libraries, for example, use 360 Search.

Here at UCSF, we are among the libraries that have built our own federated search. Twice.

Continue reading Introducing Amalgamatic

On Metrics

Collecting metrics is important. But we all know that many metrics are chosen for collection because they are inexpensive and obvious, not because they are actually useful.

(Quick pre-emptive strike #1: I’m using metrics very broadly here. Yes, sometimes I really mean measurements, etc. For better or for worse, this is the way metrics is used in the real world. Oh well.)

(Quick pre-emptive strike #2: Sure, if you’re Google or Amazon, you probably collect crazy amounts of data that allow highly informative and statistically valid metrics through sophisticated tools. I’m not talking about you.)

I try to avoid going the route of just supplying whatever numbers I can dig up and hope that it meets the person’s need. Instead, I ask the requester to tell me what it is they are trying to figure out and how they think they will interpret the data they receive. If pageviews have gone up 10% from last year, what does that tell us? How will we act differently if pageviews have only gone up 3%?

This has helped me avoid iterative metric fishing expeditions. People often ask for statistics hoping that, when the data comes back, it will tell an obvious story that they like. Usually it doesn’t tell any obvious story or tells a story they don’t like, so they start fishing. “Now can you also give me the same numbers for our competitors?” “Now can you divide these visitors into various demographics?”

When I first started doing this, I was afraid that people would get frustrated with my push-back on their requests. For the most part, that didn’t happen.

Instead, people started asking better questions as they thought through and explained how the data would be interpreted. And I felt better about spending resources getting people the information they need because I understood its value.

Just like IT leaders need to “consistently articulate the business value of IT”, it is healthy for data requesters to articulate the value of their data requests.

Headless JavaScript Testing, Continuous Integration, and Jasmine 2.0

Earlier this month, my attention was caught by a short article entitled “Headless Javascript testing with Jasmine 2.0” by Lorenzo Planas. Integrating our Jasmine tests on Ilios with our Travis continuous integration had been on my list of things to procrastinate on. The time had come to address it.

After integrating Lorenzo’s very helpful sample into our code base, we ran into a couple of issues. First, the script was exiting before the specs were finished running. The sample code had a small number of specs to run so it never ran into that problem. Ilios has hundreds of specs and seemed to exit after running around 13 or so.

I patched the code to have it wait until it saw an indication in the DOM that the specs had finished running. Now we ran into the second issue: The return code from the script indicated success even when one of the specs failed. For Travis, it needed to supply a return code indicating failure. That was an easy enough patch, although I received props for cleverness from a teammate.

I sent a pull request to the original project so others could benefit from the changes. Lorenzo not only merged the pull request but put a nice, prominent note on the article letting people know, even linking to my GitHub page (which I then hurriedly updated).

So, if you’re using Jasmine and Travis but don’t have the two yet integrated, check out Lorenzo’s repo on GitHub and stop procrastinating!

Improving Code Quality: Together. Like We Used To. Like a Family.

We had a day-long session of hacking trying to improve the code quality and test coverage of Ilios.

This post is clearly not a step-by-step instruction manual on transforming an intimidatingly large pile of spaghetti code into a software engineering masterpiece. I hope video is posted soon of Dan Gribbin’s jQuery Conference presentation from last month to see what wisdom and techniques I can steal acquire.

In the meantime, here are a few small things we learned doing the first session.

  1. Give everyone concrete instructions ahead of time regarding how to get the app set up and how to get all the existing tests running. Have new folks arrive early for a get-setup session. This allows new folks to hit the ground running and let’s experienced folks begin coding or help people with interesting problems, rather than help people just get started.
  2. Decide on a focus ahead of time. Try to fix one class of bugs, write one type of test, complete coverage for one large component, or whatever. This allows for more collaboration as people are working on similar things.
  3. Do it often or you lose momentum! I suspect that weekly is too often. We’re trying once every two-to-three weeks.

P. S. If you recognized the reference in this post’s title, then clearly you have all the skills required to work here. We’re hiring a Front-End Engineer and the job is so good that HR pasted the description twice into the ad. Submit your résumé and knock our socks off.

Running Behat Tests via Sauce Labs on Travis-CI

We use Behat for testing the Ilios code base. (We also use PHPUnit and Jasmine.) We started out using Cucumber but switched to Behat on the grounds that we should use PHP tools with our PHP project. Our thinking was that someone needing to dig in deep to write step code shouldn’t have to learn Ruby when the rest of the code was PHP.

We use Travis for continuous integration. Naturally, we needed to get our Behat tests running on Travis. Fortunately, there is already a great tutorial from about a year ago explaining how to do this.

Now let’s say you want to take things a step further. Let’s say you want your Behat tests to run on a variety of browsers and operating systems, not just whatever you can pull together on the Linux host running your Travis tests. One possibility is Sauce Labs, which is free for open source projects like Ilios.

Secure Environment Variables

Use the travis Ruby gem to generate secure environment variable values for your .travis.yml file containing your SAUCE_USERNAME and your SAUCE_ACCESS_KEY. See the heplful Travis documentation for more information.

Sauce Connect

You may be tempted to use the Travis addon for Sauce Connect. I don’t because, using the addon, Travis builds hang (and thus fail) when running the CI tests in a fork. This is because forks cannot read the secure environment variables generated in the previous step.

Instead, I check to see if SAUCE_USERNAME is available and, if so, then I run Sauce Connect using the same online bash script (located in a GitHub gist) used by the addon provided by Travis. (By the way, you can check for TRAVIS_SECURE_ENV_VARS if that feels better than checking for SAUCE_USERNAME.)

The specific line in .travis.yml that does this is:

  - if [ "$SAUCE_USERNAME" ] ; then (curl -L https://gist.github.com/santiycr/5139565/raw/sauce_connect_setup.sh | bash); fi

Use the Source, Luke

Now it’s time to get Behat/Mink to play nicely with Sauce Labs.

The good news is that there is a saucelabs configuration option. The bad news is that, as far as I can tell, it is not documented at the current time. So you may need to read the source code if you want to find out about configuration options or troubleshoot. Perhaps it’s intended to be released and documented in the next major release. Regardless, we’re using it and it’s working for us. Enable it in your behat.yml file:

default:
    extensions:
        Behat\MinkExtension\Extension:
            saucelabs: ~

Special Sauce

We keep our Behat profile for Sauce in it’s own file, because it’s special. Here’s our sauce.yml file:

# Use this profile to run tests on Sauce against the Travis-CI instance
default:
    context:
        class: "FeatureContext"
    extensions:
        Behat\MinkExtension\Extension:
            base_url: https://localhost
            default_session: saucelabs
            javascript_session: saucelabs
            saucelabs:
                browser: "firefox"
                capabilities:
                    platform: "Windows 7"
                    version: 26

Note that we configured our app within Travis-CI to run over HTTPS. In a typical setting, you will want the protocol of your base_url to specify HTTP instead.

Here’s the line in our .travis.yml to run our Behat tests using the Sauce profile:

  - if [ "$SAUCE_USERNAME" ] ; then (cd tests/behat && bin/behat -c sauce.yml); fi

Of course, if you’re using a different directory structure, you will need to adjust the command to reflect it.

That’s All, Folks!

I hope this has been helpful. It will no doubt be out of date within a few months, as things move quickly with Behat/Mink, Sauce Labs, and Travis-CI. I will try to keep it up to date and put a change log here at the bottom. Or if a better resource for this information pops up, I’ll just put a link at the top. Thank you for reading!

Pickyfill: Offline caching for picturefill responsive images

Responsive images and offline application caching do not play well together. (For an explanation, see “Gotcha #6” of Jake Archibald’s application cache article with a moderately NSFW title.)

Pickyfill (partially) solves the problem using Scott Jehl’s picturefill as a starting point. Pickyfill stores responsive images as data URLs in LocalStorage. If your page is using the HTML5 offline Appcache, pickyfill will detect this and store picturefill images as they are loaded. It will only store the images that your device displays, so (for example) an iPhone will only cache iPhone-sized images; it will not download and store crazy large images designed for large screens.

Pickyfill makes the cached images available if the user is offline. It can also improves page load time if the user is on a slow network.

Support, or Where Won’t This Work?

Pickyfill requires ApplicationCache, LocalStorage, and Canvas. If a browser that does not support these features visits a site that uses pickyfill, then pickyfill will do nothing and the experience will gracefully degrade to straight-up picturefill.

Browser Version Support
Internet Explorer 10.0+ full support
Internet Explorer < 10.0 degrades to regular picturefill
Firefox 3.5+ partial support*
Chrome 4.0+ full support
Safari (OS X, Windows) 4.0+ full support
Safari (iOS) 3.2+ full support
Opera 10.6+ full support
Android All versions it’s complicated**

*In Firefox, pickyfill will cache images on load, but not on resize. This is to avoid caching truncated/corrupted images. On resize, behavior gracefully degrades to regular picturefill behavior.

**Android browser in Android 2.3 (and probably others) does not implement toDataURL() completely/correctly. Pickyfill will detect the problem and degrade gracefully to regular picturefill behavior. Otherwise, pickyfill is fully supported in Android.

How do I use it?

Use picturefill the same way you would without pickyfill. The only changes will be to load pickyfill.js after picturefill.js and to give your site/page an HTML5 Appcache manifest.

<html manifest="/manifest.appcache">
    <head>
...
        <script src="/assets/js/matchmedia.js"></script>
        <script src="/assets/js/picturefill.js"></script>
        <script src="/assets/js/pickyfill.js"></script>
...

Although it would be better to minify and concatenate the JS files, the above code is for clarity.

Because pickyfill will only cache images that are actually displayed, it is possible for a user to visit the site, then visit the site again while offline, resize their browser, and end up with a broken image (because the image that is required at the new browser size was never downloaded before and therefore has not been cached). For this reason, it is important to have an appropriate small FALLBACK image in your offline Appcache.

FALLBACK: imgs/ imgs/fallback.png 

Demo URL: http://trott.github.com/pickyfill/
Project on GitHub: https://github.com/trott/pickyfill