Metric App // UX Research + Design

Role

Product Intern at Emma, Inc.

Project Type

Full-scale usability test on Emma's iPhone app, Metric, for its summer update

Team Members

Conducted individually under the supervision of the Client Insights Lead + Director of Product

Background

This summer, I was an intern at Emma, an email marketing company in Nashville, TN. Though I had my hands in over 30 different projects this summer, one large project that I conducted from beginning to end was the usability testing of Metric, Emma's iPhone app. Our clients send beautiful email campaigns through Emma's web application and check their response data on the web app as well. However, many clients are busy and want that data on-the-go. That's where Metric came in. Metric was released in early 2014, but was due for a summer update.


The Task

The goal for Metric as an application was not to replace the response page on Emma's website, but to surface the right level of data at the right time to help Emma's clients make informed decisions and stay updated on their email campaigns. The app had already rolled out for users, but based on client feedback, some updates would be needed. Under the supervision of my mentor, Hana (Crume) Elliott, I designed and implemented a full usability study for Metric's update.


Solution

Based on the feedback from our study, I presented salient findings to the UX Design team and Product team. I also presented findings to our Marketing team for their continued learning on how to promote the app to users. Changes to wording and interactions were implemented based on the results of my study. I also designed a Community post for Emma's website that was designed to gain additional, anecdotal evidence from clients to facilitate our learning and further help clients have the best experience they possibly could with Metric and in turn, Emma.

metric open screen

THE PROCESS

Study Design


In order to better understand the Emma clients and their needs when it comes to email response data, I first went through six months of user research conducted by my mentor. I created a mini affinity wall to draw out themes and took these findings to the Product team and other business areas. I also looked at analytic data from Flurry to see where our users were spending the most time in our application. With these themes and data in mind, I designed a plan for usability testing on Metric.

emma themes
The Basics

Though I can't share our entire process, the two goals I had for testing were:
  1. Gather actionable feedback from clients to inform Metric product team on future roadmap and validate current feature set
  2. Better understand how clients are using Metric to help marketing team position the app

I accomplished these goals by:
  1. Conducting client interviews
  2. Designing the script with tasks for usability testing
  3. Piloting the test with in-house Android users (Emma staff) who were unfamiliar with Metric
  4. Modifying the script and tasks based on feedback from 8 pilot tests
  5. Recruiting local users who fit our target demographic in the Nashville area, as well as users outside of the area for testing
  6. Facilitating and completing 6 user tests, including one remote test in Portland, Oregon
  7. Compiling, analyzing, and presenting the findings to the UX, Product, and Marketing teams

Remote Testing


Though we had constraints in conducting this usability test for Metric, we attempted to recruit target users from outside of the Nashville area. Emma has a satellite office in Portland, Oregon. I came up with a plan that would allow a willing user (or users) to come into the Portland office and complete a user test for us via webcam. We had looked into other options as an alternate to this plan - such as installing a user testing app into the iPhone application itself - but this method was the most beneficial for our users. This was the first time Emma had tried remote testing, and the entire process went well.


We set up an entire system of cameras and utilized Google Hangout to be available on video for the user. We did a pilot test with an Emma staffer in Portland first to work out any kinks. The setup allowed us to see the user's face for analysis, as well as the user's interactions on the mobile phone. We also were able to record the entire session. The photo to the right shows the slightly complex, yet efficient and valuable setup.

remote testing portland

Final Update

metric old screen
metric tested screen
metric new screen

Based on the feedback we received from testing, the UX team took their redesign that we tested and made adjustments. We recommended updating features inside the app such as screen labels and adding clickable graphs to match user expectations. This redesign of the redesign went through many iterations where my mentor and I gave feedback throughout the process.


The first version of Metric released to users was the image on the far left. The second version in the middle is the designed update that we tested for users. The third and final version is on the far right.

Emma Extras

Curious about what else I did at Emma this summer? My coworkers wrote some testimonials for me to read on my last day in Nashville.


Click through the serious and fun quotes to see what everyone had to say about my work on Metric, Product, and Disney trivia.


Still curious? Here's an infographic about my summer that I made for Advanced Graphic Design.


Visit Another Project