You only need five users to run valuable usability testing.
According to Jakob Nielsen, testing more users usually doesn’t lead to more insights. But even those 5 people will probably have very different personalities, and as a moderator, you have to be prepared to deal with all of them during each usability testing session.
It’s easy to forget about the normal, everyday people who are going to be using our apps when we’re sat in our lovely office surrounded by people who live and breathe technology.
At Apadmi, we’re always collaborating with each other and getting feedback from our peers, but everyone’s at least a little biased. Even those who don’t directly work on projects are pretty tech-savvy – it’s amazing how much knowledge you absorb just from being in an environment full of people who are enthusiastic about apps and technology!
That’s why usability testing is so important. It’s a chance to get answers to our questions from people who have different experiences and different levels of confidence when using apps.
The ideal user has lots of opinions, but also a good balance of positive and negative feedback. Plus, they aren’t afraid to tell you what they’re doing or thinking, but they don’t waffle on too much.
A few minutes into a usability testing session, you can usually tell what kind of person they’re going to be and what challenges you might face over the next hour.
There are a few distinct personality types that it’s helpful to recognise early on, so you can adapt the way you run the session to get the best out of it. Here are a few examples:
The ‘People Pleaser’
A high proportion of users will probably fall into this category. Most people want to help and they think that the way to do that is just to be really nice and polite, telling you what they think you want to hear.
If you ask them open questions like ‘What do you think about this?’, you’ll usually hear ‘Oh it’s great, I love it!’. Obviously you don’t want to lead them to say negative things if they genuinely are happy, but prompting them with questions such as ‘If you could change one thing about this, what would it be?’, might help them to start thinking a bit more critically.
This applies to all types of users – pay specific attention to what they’re actually doing, and don’t just listen to their praise and assume this means everything is fine with your design.
Some people just really love a good old natter. They’ve got a captive audience in you, the moderator, and they’re being asked to talk and so the floodgates are open!
Any question you ask will be met with lengthy (sometimes rambling) answers and occasionally they’ll completely go off topic. I once had a man telling me about his holiday to Japan while I was asking for feedback on a car insurance app!
Gently interrupt them and try to guide them back to the task at hand. More specific questions are better for this personality type – you don’t want to ask anything too open-ended and risk them going off on a tangent again. Also, keep in mind that you still want users to show you what they’re doing, not just to tell you. Keep pressing them to complete the task and praise them when they do.
The ‘Silent Type’
This user is the opposite of a ‘Chatterbox’. Getting them to actually say anything is a struggle. As much as we try and put our users at ease, it can be quite an awkward and intimidating experience – especially for those who are a bit shy.
It can be tempting to fill the silence by talking if you’re only getting one word answers, but don’t be afraid to keep prompting them to think out loud.
Open questions such as ‘what are you thinking?’ or ‘talk me through what you can see’, can give you an opportunity to start a dialogue with these users.
Even though it can be tough, try not to show any frustration either, as that will likely just exacerbate the problem and cause them to talk even less. These personalities often benefit from a lot of reassurance and praise, so make sure they know that you’re not trying to catch them out and that what they’re telling you is really valuable.
I’ve been very lucky to not encounter many people like this so far, but these people do exist and they will derail the sessions if you let them. You’ve brought them there to ask their opinion and that’s exactly what they’re going to give you – no holds barred!
They might use this opportunity to rant about a negative experience they’ve had with your (or a similar) product. Sometimes they get hung up on an issue with a prototype and they just won’t let it go!
All feedback is valid – we’re not just looking for people to agree with us, otherwise the whole exercise would be pointless. But in some cases, users are going to go out of their way to pick holes in something that is otherwise sound. Listen to their gripes, ask follow up questions to try and understand why they’re feeling that way, and if they keep bringing up something you’ve already covered, then assure them that you’ve made note of it.
It’s always better, where possible, to get in a moderator who’s familiar with the project, but isn’t directly involved. They’ll be less emotionally attached and will find it easier not to react to someone who seems determined to criticise everything.
Of course, it’s never as black and white as this in real-life usability testing sessions.
Someone that might start out as shy and quiet might loosen up and become super chatty by the end of the session.
It’s a fine line to walk – you need to be flexible, and adapt your script on the fly to fit each personality.
Just always ensure that tasks are being completed so you stay on track – protect the integrity of the test and don’t run out of time!
About the Author
Lizzie is one of our digital designers; she’s previously worked on a range of mobile projects including our NHS Donorpath App.
“As a digital designer, I’ve been involved in a number of usability testing sessions during my career so far. I’ve been on both sides of the glass; sometimes guiding users through tasks, while at other times, I’m in the room with our clients capturing their insights.
“No two usability tests are the same, and each one will no doubt turn up something unexpected you’ve never seen before – like staying behind for 10 minutes after one test to talk a very nice lady through how to add train tickets to her Apple Wallet!”