5 Users is Enough For Usability Testing

This video explains how 5 users can be sufficient for usability testing if they share similar profile characteristics and engaged in scenario-based tasks with the interface being tested.

At UX Firm we believe in the power of small studies. When done right they can produce real consistency in user experience and give you the information you need to improve your interface right away without much time or budget.

How do we get to this place where we feel confident that with only 5 participants we are able to do quality research that gives us action items that we believe in? We need to go back to the the 1990s and to the work of three separate researchers. James Lewis, Robert Virzi and Jakob Nielsen were separately researching how many users it takes in order to get to some confidence in the results that you get.  All three of them separately confirmed that with about 5 users you’ll get about 85% of the results of a specific usability test, if certain conditions are in place.

This is the part I think most people may not understand, so I want to review what those conditions are. The first condition is that you’ve got to recruit very carefully from a small subset of your user population so that all 5 people in your user subgroup have similar knowledge of or lack of knowledge of your domain or the technology that will be involved.

The second condition is that your moderator’s guide needs to be very specific in helping users understand tasks within scenarios so that they are all attempting to do the same things at the same places in your interface.  When you have specific users working within specific areas of your interface, you will then find very quickly that after about 2 or 3 users you’re seeing the exact same problems.  You’re beginning to predict what will happen with user number 4 and user number 5, and you’re ready to take those findings and turn them into action. That’s the power of small studies.

What if you have a budget for more, or you want the confidence of being able to see more than one subgroup of your user population? Let’s say that you want to look at ten users.  You could take two subgroups of your user population with five in each subgroup, and you would then see overlapping findings consistent across whatever subgroup of your user population is working with your interface.  But you might also see unique findings to one particular subgroup that are not shared by the other subgroup. What if you had a budget for 12 users? In that case, you could go to smaller numbers in each of your subgroups so that you could have, let’s say, three groups of 4 participants each for your twelve or even four groups of 3 participants each in each subgroup.

When you have more participants, you can then go to smaller numbers so that you’ll be able to see the consistency across the subgroups as well as the unique findings for each subgroup.

What if you have a budget for 15 people?

Here’s where we might recommend that you don’t use all of that budget for a single study. It would be much better in many cases to test with 5 participants for your first round. You then pull those findings out, make recommendations, and they get applied to the iterative development of the product. You then test again with five more from the same subgroup or a different subgroup, and then finally test again with the third subgroup or the same subgroup. Your total of 15 participants is spread out over the development cycle so that you’re building user experience into the product as it moves through development.

The power of usability testing with small numbers

The power of usability testing with small numbers is not only great in agile development methodologies or for providing quick action to be able to use immediately in the development of your product, but if you’ve never done usability testing before it’s a way to convince your company or your product team or whatever group you’re working with that you get a lot of bang for buck with a very small study of only five users.

What sample size do you need for UX research?

is a question a lot of people have. If you’ve never done usability testing, see if you can wrangle support for five users and then tell everybody what you learned. Share the findings and you will very likely see that it’s a transformative experience where more and more testing will be demanded because of the power of small studies.