Heatmapping User Actions

A few weeks ago, Test Pilot released a study called “Firefox Desktop Heatmap 2012.” This is a update of a study that was done in 2010 to measure how frequently our users interact with various elements in the Firefox desktop interface. The study ran for one week and we received data from over 215,000 users.

Heatmaps highlight mismatches between UI prominence and feature desirability. As part of the Australis redesign, this study was designed around these questions:

What features do people use in the current layout?

What awesome Firefox features are people missing, because the UI doesn’t make them prominent, attractive, or understandable?

What features are too prominent given their usage?

Mapping the Browser Window

The heatmap below shows the percentage of users (all OSX, and Windows users with menubar enabled) who clicked on each browser element least once.

Heatmap for Firefox Desktop – For Users with Menubar Shown

Searchbars, Urlbars, Search Suggestions Are Frequently Used

Many people use the urlbar (82%) and the searchbar (46%) — 27% use search suggestions, and 60% use urlbar completions. A forthcoming article will explore how people use each, and how they overlap.

Users Use Multiple Search Engines

Many more people click on the “Searchbar Search Engine Dropdown” (the menu that lets you change the search engine for searches in the search bar) (21%), than actually change their search engine (11%).

Why might this be?

Users could be dissatisfied with the searching options we provide for them and choose to stick with their original search engine.

Users might click on the button accidentally when meaning to go to the search bar.

We have timestamps for every action performed, so we could approximate what percentage of users clicked the menu by accident, if we assume this would be a very fast click. If this is a high percentage, then this is the wrong location for such a menu. If it turns out that users are dissatisfied with the options for search, we need to either rethink what search engines are displayed by default in this dropdown, educate users on how to add their own search engines to this menu, or restructure the way we present the search results from each search engine. (We will perform this analysis if there is interest.)

Low Panorama Usage

Of the major window elements, the least used was Panorama (the “Tabview button”), which displays tab groups (0.7%). If the users who click on the Panorama button are, indeed, utilizing Panorama heavily (which we could discover by looking at frequency of usage), a good revision might be to turn Panorama into a option that can be toggled on and off easily, or an add-on. It is also possible that users simply don’t know what Panorama is, since the button to view tab groups is shut off by default.

Bookmarks are Exposed Multiple Ways, with No Clear ‘Best’ Interface

For completeness, here is the heatmap for the same elements for those users with the menu bar disabled:

Heatmap for Firefox Desktop – Menubar Hidden

We can see that the numbers are very similar to those in the previous heatmap, with the exception of the Bookmarks Menu Button (always shown on OS X, but on Windows only when the menu bar is hidden). This button is relatively popular (19%).

Bookmarks are overdue for simplification and redesign. As explored in the next section, there is no obvious “right” answer to how to expose Bookmark functions, and different people seem to use different access points. If the users who were clicking on “Bookmark This Page” in the menu were only doing so because the menu bar was enabled and this menu button was not available, maybe we should think about keeping this button always shown. We need to see if there is a preference for the menu bar or the desktop button when both the bookmark menu bar and menus are available (this is only true in OS X). If there is a strong preference for the shortcut button, perhaps the bookmarks menu should disappear altogether.

Exploring the Menu Bar

This heatmap summarizes the percentage of users (OSX and Windows users with visible menu bar) in the study who clicked on various menu items at least once during the study:

The menu items used by the most people were:

Personal Bookmarks (25%)

Recently Visited Pages (16%)

Bookmark this Page (15%)

Open Downloads (10%)

Clear Browser History (10%)

Print (7%)

Save page (6%)

Open Addons (6%)

Open New Tab (5%)

Using Menus instead of Browser Chrome

Two of these items – “Bookmark This Page” and “Open New Tab” – have shortcuts buttons in the browser window. Why are users not using these always visible shortcuts, which are arguably more accessible than the menus? There are two immediate hypotheses that come to mind: users only click on the menu options infrequently, and generally use the shortcut buttons, or there are users that only use the menu, and prefer not to use shortcut buttons at all.

By looking at the frequency with which these options are used, and not just the percentage of users that utilize each option, we can gain more insight into these possibilities. If a nontrivial percentage of users never use these shortcuts and are still using their menu equivalents frequently, we should turn to either a survey or interview approach to find out why. It might turn out that the shortcut buttons are not clear enough, in which case we’ll have to turn to education about using these features or clarifying the UI itself. If users simply prefer using the menu items, we need to discover why this is.

For the rest of these higher-frequency items, if we added shortcut buttons, would this benefit the user, or merely clutter up the space? Would more users know about these options if we added a shortcut button? Unfortunately, there is no way to determine the answers to these questions with the current set of data we have amassed. We would need to do some A/B testing or interviewing to see what the impact of such buttons would be. If only a small percentage of users (>2%) would use such desktop buttons with a consistently high frequency (maybe more than once a day, for example), a good route would be to develop an addon to add these buttons to the main browser window. If such an addon is successful, it would indicate that this is a feature that could be integrated into Firefox.

Rethinking the Help Menu

Looking at specific menus suggests other simplifications. Of people using the help menu, most only click on the generic “Firefox Help” item. Are users so satisfied with “Firefox Help” that they don’t need to click on these other menu items? Or do they only go to the “Help” menu when they need exactly that – help – and not expect to find categories such as “Submit Feedback” there? If users are getting help through external sources that are inferior to what the Firefox help system can offer, then the UI needs to be clarified to encourage users to use this system. Promoting “Reset Profile” into this menu might also improve access to that feature.

Usage Patterns are Stable From 2010

It’s also interesting to compare the results from this heatmap to the study that was done in 2010. The percentage of users that used the searchbar at all dropped from 61% to 47.5%, and the Select Search Engine Menu went from 32% to 22%. This might reflect changes in how people search on the web in general. Are users more likely to search by typing into the urlbar than the searchbar now? Do they follow links more often than they do searches? Not much has changed in the Firefox interface in the last two years to discourage searching in the searchbar, so a change in user behavior remains the most likely explanation for this shift. However, the previous study was done with a different demographic – Test Pilot users of two years ago – so it’s possible that this accounts for some of the discrepancy. It is still worth considering a Test Pilot study where we track not just usage of the urlbar, but how often searches are performed as opposed to urls entered. A high volume of searches would indicate that we could move away from having a search bar at all.

Future Research

With preliminary research like a heatmap, sometimes more questions are raised than answered. But the questions that we get that come directly from user data are the ones that, when answered, can give us some of the most valuable insights about Mozilla products. To recap, the following are some of the takeaways that we can get from the heatmap and some of the actions we should now pursue.

See how often users are performing searches with the searchbar as opposed to the urlbar, and how this has changed over time.

Figure out why users are using menus instead of toolbar buttons. See if we can characterize this group.

De-emphasize Panorama to an addon, unless a usability study can be done that demonstrates enthusiasm for it once users know of its existence.

Discover why users aren’t using the help menu and figure out how to get users help when they need it.

Examine user Bookmark feature entry points and evaluate plans for unifying them.

Determine why search engine dropdown menu is clicked and then think about its position near the search bar and/or the search options that are enabled.

Have any other ideas about the data? Leave them in the comments below!

(authors: Ilana Segall, Gregg Lind)