UX Benchmark Testing:

Introduction

UX Benchmark Testing is an efficient method of studying how users interact with, and think about an interface. It’s a crucial step toward continuously improving your site’s user experience. Benchmark Tests that take place periodically act as a point of reference for your site’s metrics or Key Performance Indicators (KPI’s) to be compared and evaluated on a regular basis. The goal of this Benchmark Test is to identify any patterns that are working well or could be improved on Comfortiva.com. Running the benchmark test on a regular basis ensures the continuous improvement of the site over time, and aids in directing us to any systemic issues that currently exist and impact the customer/ user’s experience when on the site. The success of the tasks/patterns are measured by asking principally verifiable questions and analyzing the data. By asking verifiable questions we’re able to obtain measurable, qualitative data that can be easily compared from year to year as new and substantial site updates occur. In usability and benchmark testing 5 users/participants are typically tested. This small number allows discovery of as many usability issues as a larger sample would provide. A sample of 5 participants assures the test’s maximum benefit to cost ratio is reached, as testing costs increase with each additional participant and valuable/new feedback decreases with each participant. This only applies directly to usability issues; other qualitative feedback or opinions of the user are useful for insight but not necessarily representative of all users (Jakob Nielsen, Why You Only Need to Test with 5 Users, March 19, 2000). This report provides: the Benchmark findings, KPI’s (Key Performance Indicators), testing methodology and plan, as well as the participant/demographic summary.

Testing Methodology

In this Benchmark we conducted usability testing with a total of 10 participants, across one eCommerce site: Comfortiva.com. We acquired the participants with the assistance of Userlytics.com. 5 out of 10 participants took part in the benchmark test using their mobile device and navigating the responsive site, while the remaining 5 tested the site using a desktop or laptop. Participants performed a total of 5 key tasks while navigating the live site. Userlytics recorded the participants’ behavior using screen and audio recording/ sharing software. 10 out of 10 participants spoke aloud throughout the administered test, guiding us through the tasks in order to provide an in-depth understanding of their actions and answers. The tests administered were unmoderated, and participants were given written tasks to read aloud and follow unaided.

TEST PLAN:

Benchmark questions/tasks will remain as consistent as possible throughout test reiterations. Tasks have been decided on and created with the critical goals of the website in mind. Edits made to tasks, such as verbiage, may impact results from test to test and must be taken into consideration when comparing data. The tasks remain consistent throughout devices/platforms, which allows us to ensure users are having a congruent experience on both the mobile and desktop site.

PARTICIPANT INTRODUCTION:
Participants were introduced to the benchmark study by being asked to imagine: “You need to find a new pair of everyday shoes for work. You decide to shop for them online, please begin your shopping experience at Comfortiva.com if not already directed there.” This introduction aids in moving the participants into the right mindset before entering the website, and prepares them for the upcoming tasks ahead.

Task 1: INITIAL IMPRESSIONS
Participants were asked to examine the Comfortiva.com homepage, and provide us verbally with their initial impressions of the site. This task typically took no longer than a couple minutes. There were no restrictions on what participants could 7 comment on, and answers ranged from design critique, remarks about information architecture and hierarchy, as well as explanations on what feelings or emotions the site evoked at first glance. After reviewing the homepage participants then gave the 3 words (or phrases), that summed up their impressions. These tasks allow us to examine how the brand’s site is perceived by users and customers. Whether the site evokes what the brand represents, whether customers find it trustworthy, what their overall impression of Comfortiva’s online presence is.

TASK 2: BUY A PAIR OF SHOES
We asked participants to start this task from the homepage and find a specific pair of slingback heels that were selected at random (The Faye in the color red). The only constraint on how the participants completed this task was that they could not use the search function. Once the heels were found, the participant was asked to proceed through checkout and complete the order, using test credentials given. This task allows for us to examine the findability of a specific product on the site as well as the flow from product page to shopping cart/check out. We’re able to observe how easy it is for the users to navigate the website, use the product filters, and purchase a product.

TASK 3: ORDER STATUS
Task 3 requested participants to find on the website where they could check their order status, and to then check on the status of the Faye that they purchased in Task 2. Participants were requested to begin this task from the homepage. This task provides insight as to how easy and fast a customer is able to find the order status of their recent purchase, which is a standard but critical task.

TASK 4: PRODUCT DESCRIPTION
Participants found a pair of shoes in task 4, that they could imagine wearing to a friend’s wedding. They were also asked to rate the product’s page and available description. Participants were not permitted to use the search option to find the shoes, however they could begin the task from the page they were currently on.
Task 4 serves the purpose of examining how customers/users browse and find non-specific products on the site, as well as inspect how the product page and 8 product descriptions are performing. Some information we gathered was whether it was easy to browse the site, if the user has any frustrations, how valuable the product description is to the customer and if anything is missing that’s valuable for the customer to know prior to purchase.

TASK 5: INFORMATION FINDABILITY
In this task we ask the participant to focus on finding important information, specifically the site’s return policy and the customer service phone number. This information is key for online consumers and we’re able to see how long it would take a new customer to find this information if they were ever in need of it, as well as the ease of findability.

TASK 6: EMAIL SIGN-UP
In this task we ask the participant to find where on the website they can sign-up for emails and complete sign up with a fake email address. This task provides insight as to how easy and fast a customer is able to sign-up for promotional emails.

PARTICIPANT LIST/DEMOGRAPHIC
The participant demographics selected for the benchmark tests were chosen based on Comfortiva’s data collected by Google Analytics. The participants themselves were screened by userlytics.com, the site used to administer/record the benchmarks.