7 Secrets to Designing a Great Usability Test

Seven Moais looking towards sea in Easter Island

Usability testing is a great way to get a close-up look at your audience’s perceptions and expectations, but if you don’t have a lot of experience setting up user tests it’s easy to make mistakes or simply leave things out. Doing so undercuts the value of your testing and wastes both your time and that of the participants.

A poorly designed usability test seriously impacts the results you’ll get in two ways: users struggle to correctly carry out the tasks in your user flow, and you get less insight into relevant usability issues with your application. When your testers don’t understand the tasks, aren’t prepared to perform them properly, or misinterpret the instructions, your research doesn’t pay off the way it should. There’s a good chance you won’t learn what you were hoping to learn about your interface.

Here are 7 tips from our own extensive experience running usability studies to help you design a user test that will avoid common errors and maximize the knowledge returns you get out of your usability research.

1. Create an Engaging, Immersive Scenario

To get the best feedback from your user tests, the tester should be immersed to the fullest extent possible in the mindset of somebody using your product in a real-life context. To accomplish this, design a scenario for your testers that is detailed and realistic. Write a scenario that sounds like a story, not a set of instructions.

An ineffective scenario, for example, might sound like this: “You need to buy renter’s insurance and want to explore your options.” A great scenario would be: “Your friend just paid thousands of dollars to repair damage caused when a guest accidentally started a kitchen fire. You have guests over often and want to be covered in case something like this happens.”

This allows the tester to really dive into the scenario and explore your product through the perspective of a real user. If you’ve done your demographic selection well, a detailed scenario like this one may even be identifiable to your testers.

2. Design a Journey, Not a To-Do List

Although your testers will be guided through your application by clearly delineated tasks, you should mimic a real-life user journey as much as possible. Think about how someone visiting the site might progress through different pages and phases. You probably have a pretty good idea of how visitors move through your site and the actions they take; use that knowledge to lay out a smooth and natural journey that a real user would actually take.

A task need not involve a concrete action like registering or making a purchase. Sometimes asking the tester to browse around or explore for a bit, whether in a specific section of the website or across the whole thing, makes sense at that part of the journey. Leaving them to navigate and make independent judgment calls can tell you a lot about the way people process what they see and how they’re approaching the content of your site.

3. Be Like Their Inner Voice, Not Their Drill Sergeant

Writing a successful task is not just about choosing the right steps of the journey; wording and tone matters a lot too. Don’t write orders for your testers to follow. Instead, write pieces of a thought process.

Draw on the way you that yourself think when you visit a new website. You probably talk to yourself, whether out loud or in your head – “Now, where is the Search box?” “How can I see the newest stuff they put up?” These are natural ways of processing a site, and imitating them helps to shape a more natural experience in which the test fades into the background. Instead of “Read about the products and services offered,” try “What services does X offer? Is there a plan that will cover kitchen accidents that’s in your budget?”

4. Write for First-Time Visitors

Being so intimately familiar with your own product makes it easy to inadvertently talk about it in a way that doesn’t make sense to new users. Remember that your testers probably know nothing about your product and how it works. Steer clear of industry terms or brand-speak; these are likely to just confuse your testers. Sometimes the result is a tester that mistakenly believes he has completed a task and moves on without ever having done the action you intended to test.

When you write your tasks, use simple and generic wording and think hard about what will and will not make sense to first-time visitors.

5. Structure One Objective Per Task

Don’t be afraid to break things down – if one of your tasks has multiple discrete parts, especially parts that are contingent on each other, it might be better to divide them up. You don’t want to give your testers too many hints by spelling out the process, or confuse them with references to things that don’t happen until the next page.

Additionally, by limiting each task to just one small goal, you can get more specific, targeted feedback – for example, ask your testers in one task “What do you think the ‘Links for You’ tab is for?” and then, in the next task, “Click on ‘Links for You.’ Were you right?”

6. Supply Information as Tasks Demand It

Many people make the mistake of putting all the information their testers may need during the test into the scenario. If your testers will need to enter a test credit card number, or there’s a certain name and phone number you’d prefer them to use for forms, don’t stuff it all into the scenario, even if they can re-read the scenario at any time during the test. This is just distracting, and they may forget about it by the time they need it.

Space it out: when their task is to complete the purchase of a new bath towel, give the card information in that task. When their task is to schedule an appointment, supply a fake address that they should use.

7. Don’t Lead the Witness

Your word choice when writing tasks should avoid key words used on your application – tell testers the end goal to be achieved, not the action to take. For example: “Save a product you like so you can come back to it later” instead of “Add an item you like to your wishlist.” Not only will this show you how easily the user identifies and locates the way to complete the task unaided, it might reveal a totally different way they might think about achieving that goal – maybe their first instinct is to bookmark it on their browser. What you really want to get out of your testing, anyways, is how people are really using the product.

The bottom line is, if the words you use are the same as the words that appear on your site, the interactions you see in your results won’t be genuine; you’re effectively giving the tester the ‘answer.’ Refrain from leading the witness, and you’ll learn a lot more.

In Conclusion

There are many more ways to optimize your user research, but with these 7 secrets you will be able to avoid the main pitfalls and get valuable, relevant usability feedback. It’s worth the extra time it takes to design a great user test.

Tim Rotolo is a UX Architect at TryMyUI, a website usability testing service. You can follow TryMyUI on Twitter @trymyui.


Speak Your Mind