Demystifying the art of giving user research feedback

A comprehensive playbook on how to review user research plans, sessions, and reports (including your own)

Over the past few years, I’ve spent quite a bit of my time giving other researchers feedback on their work. In fact, if you were to ask my lead, he’d tell you I’ve spent entirely too much time reviewing other people’s studies. Every once in awhile though, he’d nudge me to think of offloading some of that work onto others on my team. To pass the torch in a way, by teaching someone else those skills and moving on to new challenges myself.

Though I heard (and agreed) with what he said, I didn’t act on his suggestion. True enough, the more time I spent down in the weeds of other people’s research, the less time I had to pull back, see the bigger picture, and tie all their stories together. And the more time I spent reviewing work, the more I was taking away opportunities from others on my team who maybe wanted to learn how to do it themselves. But the problem was I didn’t know how to teach others something I thought came innately to me. I’d been looking at other researchers’ work for what felt like a very long time, and therefore thought I had developed some sort of instinct for weeding good studies from bad ones. “It just comes to me” I’d respond, thinking it probably wasn’t possible to abstract my way of thinking into a teachable skill.

But, as the requests for feedback continued to metaphorically pile up on my desk, my approach seemed more and more unsustainable: I knew something needed to change. And so I began to pay more attention to what was actually going through my head as I reviewed other people’s work, attempting to break down my own process. Eventually, it dawned on me that the answer was far simpler than I imagined: questions. All my brain was doing whenever I looked at research documents or sat in on sessions was looping through a series of questions. In a way, it seems predictable (and fitting) for a researcher to structure their thoughts entirely in questions, and so once I began writing them down, I realized that this skill was easily transferrable.

I know now that being able to give good feedback isn’t fully contingent on having years’ worth of exposure to different types of research projects. It’s not some sort of mystical or innate ability that can’t be abstracted or taught. And it’s not something that should fall solely on one person’s shoulders, but rather a shared responsibility that can help entire teams level up their skills together. And so, we put it all into this playbook.

Today, it’s being used at Shopify, not only by research managers looking to give their reports better feedback, but also by the reports themselves, as they look for opportunities to mentor one another. And, if you look closely enough, you’ll see it also serves as a checklist of sorts: questions researchers should be asking themselves about their own work before shipping it.

As we align those who produce the work and those who review it more closely on a common set of questions, we can tighten the feedback loop and collectively shorten the time to get quality research out the door.