Get a baseline performance measurement

First of all, I need an indication of the performance of the existing design when it comes to customers finding and navigating to the ‘Lists & Recipes’ section.

To find this, I created a click test on the UsabilityHub platform. Participants are asked “Where would you click on this site to find some inspiration and ideas for meals?”, and then shown a snapshot of the YourGrocer site.

The platform then records when and where the test participants click to complete the task. For this test, the ‘Lists & Recipes’ navigation tab in the green bar is the success target.

Here’s what the results of our first test look like — and it turns out that the current baseline design is performing fairly decently!

I created the snippet below on the results page by selecting the successful clicks. This gives us a simple bar chart showing how well the baseline design performs in terms of participant accuracy:

“Area #1” in this chart represents the success target. The duration is mean time-to-click.

78% accuracy is a good starting point so I might not see too much of an improvement in this regard. I’m not setting a specific goal around this metric, but it’s still a key number to keep an eye on.

Time-to-click results

When looking to optimize elements in this way, pay attention to the change in mean time-to-click for your successful responses.

The mean gives you a broad indication of task speed, but to get a more detailed look at the spread of the responses, creating histograms from the raw test data is required.

Time-sensitive data like this will be simpler to analyze from histogram charts when compared to just using the numbers, particularly if your audience doesn’t have a deep knowledge of statistics. They’re also great for presentations.

Here’s a histogram of the first test results:

This chart shows the spread of responses using 10 second buckets. 27 of the 39 successful responses happened in the first 30 seconds of the test, which is what I want to see.

However, the tail of responses that continue into the 30–60 second range is concerning, and there are 3 successful responses that are much slower than that — with the slowest response taking almost 2.5 minutes. There’s definitely room for improvement here.

My goal for this optimization exercise is to improve the spread of time-to-click so that there are no responses slower than 30 seconds. I picked this number because of the shape of the data, and because it seems to be a reasonable amount of time to allow for this task.

It’s unlikely that I’ll get all the way there because there are always lagging responses in tests like this. After all, we’re using human participants here! But it’s a good goal to strive for.

Including a satisfaction metric

In the baseline test, we also included a question to measure the Customer Effort Score (CES) of the original design. CES is a great metric for measuring the impact of design changes because it’s a measure of participants’ perceived effort when completing a task.

It gives you a window into user expectations and sentiment, independent of more concrete metrics like time-to-click or proportion of successful clicks. In addition to this, CES shows teams how their work can influence customer attitudes. These factors make it a worthwhile inclusion for this kind of remote user test.

The baseline CES is a 5 (out of a possible 7), which is a decent result to start with. Let’s take a look at the shape of the responses:

CES results for the baseline screenshot.

One of the issues that I’ve found with using CES is that it moves around a lot with the sample sizes at n=50, which is my safe default for optimization tests like this.

I think CES is still useful to include, because it gives me a good check of participant sentiment without having to do free text analysis, which would be required for more open-ended questions.

Just keep in mind that it doesn’t have good resolution when you’re fine-tuning designs. Use it to check your course, not direct it. I’ll note it in the test results, but I’m not setting a specific goal to improve it.