Background

Juicebox is our SaaS platform for quickly making beautiful data-storytelling apps. These apps are designed specifically for the non-data-savvy workforce, so we have users who are adapting to features with functionality that they may not be familiar with or have never seen before.

At Juice Analytics, we partner with Fullstory, a platform that allows us to view our users’ behavior while interacting with the app. By watching users adapt to new features in our apps, we have learned a thing or two: no matter how intuitive a feature’s design is, users will click on it in an exploratory manner. This shouldn’t be a cause for alarm, we all have an innate desire to understand. What may be a cause for further investigation, is when a user, or many users for that manner, continue to click on a feature of an app repeatedly that shouldn’t theoretically be interacted with all too often.

In our case, we noticed that, in addition to exploratory clicking on our app’s features, users would click repeatedly on the title of a data visualization. The users would click on the title of the visualization and the visualization itself would collapse. After the visualization collapses, the user would most always then click on the title again, suggesting that they did not expect or wish to collapse the visualization. This creates a disruption in the fluidity of the experience of the app.

Interestingly, the persistent clicking did not occur on all titles in the stack, only ones in which metrics for the visualization were emphasized as a result of their option choice at the top of the page:

Emphasized Title (lots of clicks)

Non-Emphasized Title (very little clicks)

Perhaps even more interestingly, their clicks on the titles seemed to be targeted at the emphasized metrics:

Expected Click Behavior

Actual Click Behavior

Observing this sort of clicking behavior intrigued us and our desire to create the most fluid user experience led us to form a research question and naïve hypothesis:

Research Question: Why do users click on the title’s metrics so often in some slices but not in others?

Hypothesis 1: Users think that the emphasized metrics are buttons that will allow them to choose what metrics are included in the visualization, when really clicking on the title will only collapse the slice.

Hypothesis 2: There will be no significant differences between the comprehension of emphasized and non-emphasized titles.

Of course, as the UX research and design community is well aware, this guess (read: hypothesis), is only a guess, and anecdotal evidence can be misleading. So we reached out to UsabilityHub who were more than willing (if not excited about our study proposal) to help us by supplying participants for our study on title styles.

Method

In an attempt to not only test our research question, but also to test for effectiveness of alternative designs, we used two different tests from UsabilityHub’s test bank. The first one was the 5-Second Test. We used this test to measure people’s comprehension of the contents of the title without asking them to specifically look at the title. After looking at the interface for five seconds, participants were asked which two measures (out of six options) were indicated in the title of the visualization. The order of the options between participants were randomized in order to avoid order effects.

The second test we used was the Question Test. We used this test to measure people’s perception of what they think would happen if they clicked on the title. In this test, participants were shown the interface and were asked what they believed would occur if they clicked on the title. Specifically they were given three options:

It would allow me to choose different metrics to rank It wouldn’t do anything It would collapse the rank chart

The order of the options between participants were randomized in order to avoid order effects.

Participants & Design

We gathered a sample of 200 participants who were at least 18 years old and had completed college. These specific exclusion criteria were used to match the user base of Juicebox. Don’t forget that you are trying to use groups of participants to generalize to behavior and preferences of your own, actual users. Participants were split into 4 conditions which tested the comprehension and perception of 4 different title styles.

All participants were presented with this interface: