User Testing is not you testing the users.

It’s the users testing you.

Our responsibility as designers is to check our designs in the realistic, practical scenarios they are were designed for.

Web designers get real users to try out new ideas.

Car designers get people to sit in the car and test drive it.

Underwear designers put real asses in those things!

But user testing is not testing whether the users get it right, or if the crash test dummies die the right way, or if your ass does the wrong things.

It’s about the design.

Your job is to find patterns among the users that help you improve your design. Not to get results that make you happy.

****

There is a lot of advice out there about exactly how to do a user test, but I think your mental approach is more important than your procedure.

5 things to keep in mind:

1) Are you scoring them, or observing them?

You may come across people giving advice about how to “score” your users. That sounds an awful lot like testing the users to me. And that’s not what user testing is about.

Your mission as the tester is to find weaknesses and problems in your own design. Whether the user gets a “good score” isn’t important. Watch how they do a task. Listen to what they say or ask. If they complete the task in a way you didn’t expect, that’s interesting, not wrong. Investigate.

If you develop a “scoring” system as a way to compare results, that’s fine. Maybe even good. But score the design, not the users.

****

2) Are you watching to see where they make mistakes, or where you have mislead them?

Users can’t make mistakes in a user test. It’s impossible. They are only doing what you have lead them to do, based on assumptions you have allowed them make.

You should be looking for your own assumptions instead. If you have assumed what the users assume, they will do what you expect. But everywhere you have assumed something incorrectly, they will deviate from your plan.

Don’t blame them for screwing up your perfect plan. Realize your plan isn’t perfect, and get back to work.

****

3) Have you given them tasks to try, or procedures to fail?

Most user testing involves some sort of task or objective for the user. Stuff like “You are trying to buy a cozy new prison facility. Find one you like and add it to your cart.” In most tests, the instructions won’t be too much more complicated than that.

However, the expectations in your head can be much more complicated.

If you are watching to see if they follow the steps you expect, you’re doing it wrong. You should be watching to see how they complete the task. After all, you didn’t give them steps to complete (and you shouldn’t), you gave them a goal.

If they don’t complete the task, your design is hard to use. If they complete it in a “weird” way, take a lot of notes and ask a lot of questions.

****

4) Are you asking them a question, or leading them to an answer?

It can be very tempting to look at a user test as a “win/lose” situation for yourself. You want users to do certain things in a certain way, because it would mean your design is awesome.

Don’t do it.

Having a “preferred result” in your mind will make you ask questions in a way that is more likely to get that result. You will investigate things that agree and ignore things that disagree, or — if you’re an idiot — you might even try to nudge them toward the result you want.

The only result you want is the truth. It doesn’t matter if they love your design or hate it. In fact, if users love your design that isn’t very useful, because it doesn’t tell you how to improve anything.

****

5) Are you waiting for a particular result, or observing what happens?

Some tests are aimed at completing a task. Some are aimed at certain types of actions. If you observe a test just to see if the user completes a task or clicks a button, you have missed most of the value in the test.

Just because a user completes the task eventually, doesn’t mean the experience was good.

Bad testers will ignore the actions or comments along the way, but as soon as the user says the magic words, they light up, check the box, and move on to the next question/task.

A good way to know if you are making this mistake is to read your testing notes. Do they say “1) Completed task, 2) Completed task, 3) Didn’t complete task…”?

If you do this, you’re missing the point.

But moreover, if the user can see you do this — now we’re talking about your body language — they will quickly learn which types of answers make you happy and which don’t. Every other thought they have will be hidden from you, because they don’t think it’s “right”.

And yes, they will do that even if you say “There are no right or wrong answers.” And that’s your fault, not theirs.

****

So next time you sit someone down to try out a new design, clear your mind and relax; you’re being tested.