5 Users is Enough For Usability Testing
This video explains how 5 users can be sufficient for usability testing if they share similar profile characteristics and engaged in scenario-based tasks with the interface being tested.
At UX Firm we believe in the power of small studies. When done right they can produce real consistency in user experience and give you the information you need to improve your interface right away without much time or budget.
How do we get to this place where we feel confident that with only 5 participants we are able to do quality research that gives us action items that we believe in? We need to go back to the the 1990s and to the work of three separate researchers. James Lewis, Robert Virzi and Jakob Nielsen were separately researching how many users it takes in order to get to some confidence in the results that you get. All three of them separately confirmed that with about 5 users you’ll get about 85% of the results of a specific usability test, if certain conditions are in place.
This is the part I think most people may not understand, so I want to review what those conditions are. The first condition is that you’ve got to recruit very carefully from a small subset of your user population so that all 5 people in your user subgroup have similar knowledge of or lack of knowledge of your domain or the technology that will be involved.
The second condition is that your moderator’s guide needs to be very specific in helping users understand tasks within scenarios so that they are all attempting to do the same things at the same places in your interface. When you have specific users working within specific areas of your interface, you will then find very quickly that after about 2 or 3 users you’re seeing the exact same problems. You’re beginning to predict what will happen with user number 4 and user number 5, and you’re ready to take those findings and turn them into action. That’s the power of small studies.
What if you have a budget for more, or you want the confidence of being able to see more than one subgroup of your user population? Let’s say that you want to look at ten users. You could take two subgroups of your user population with five in each subgroup, and you would then see overlapping findings consistent across whatever subgroup of your user population is working with your interface. But you might also see unique findings to one particular subgroup that are not shared by the other subgroup. What if you had a budget for 12 users? In that case, you could go to smaller numbers in each of your subgroups so that you could have, let’s say, three groups of 4 participants each for your twelve or even four groups of 3 participants each in each subgroup.
When you have more participants, you can then go to smaller numbers so that you’ll be able to see the consistency across the subgroups as well as the unique findings for each subgroup.
What if you have a budget for 15 people?
Here’s where we might recommend that you don’t use all of that budget for a single study. It would be much better in many cases to test with 5 participants for your first round. You then pull those findings out, make recommendations, and they get applied to the iterative development of the product. You then test again with five more from the same subgroup or a different subgroup, and then finally test again with the third subgroup or the same subgroup. Your total of 15 participants is spread out over the development cycle so that you’re building user experience into the product as it moves through development.
The power of usability testing with small numbers
The power of usability testing with small numbers is not only great in agile development methodologies or for providing quick action to be able to use immediately in the development of your product, but if you’ve never done usability testing before it’s a way to convince your company or your product team or whatever group you’re working with that you get a lot of bang for buck with a very small study of only five users.
What sample size do you need for UX research?
is a question a lot of people have. If you’ve never done usability testing, see if you can wrangle support for five users and then tell everybody what you learned. Share the findings and you will very likely see that it’s a transformative experience where more and more testing will be demanded because of the power of small studies.
UX Toolkit – UX Research Methods – Part 1
3 Popular UX Research Methods – Usability Testing, Card Sorting & Heuristic Review
Carol Barnum explains usability testing, card sorting & heuristic review, 3 popular UX research methods in the UX Firm toolkit.
Want to learn more about usability testing? Check out the questions we get ask most frequently about Usability Testing.
UX Toolkit – Top UX Research Tools – Part 2
UX Research Tools: Diary Studies, Contextual Inquiry, Focus Groups
This video continues our discussion on some of the popular tools in the UX research toolkit. In part 2 we provide tips on when and how to use diary studies, contextual inquiry, and focus groups.
In Part 1 we covered usability testing, heuristic evaluation, and card sorting. And whereas card sorting is a great tool to use in early development of your product or interface, diary studies are a great tool to use very late in development or even after launch.
This is because diary studies provide a way to get feedback on user experience over time. They are called longitudinal studies because they take place typically over a week or two in which users are prompted with questions in which they respond during their day to specific types of prompts. Users can provide pictures, they can provide text, or they can provide chat, depending upon how the diary study is set up. They give great insight into the way in which a user engages with the product or the interface as he or she is learning it over a period of time and also insights into the environment in which the user is engaging with the product. So, diary studies are a great tool for learning about the immersive experience of users.
Another tool in our UX research toolkit is called contextual inquiry or sometimes ethnographic research. This is a great tool to get information about your users and their environment even before the product is designed because it works through the process of shadowing somebody in the environment in which they’ll be using the product or interface or interviewing them on site where they work or play, wherever their environment is. You gain great insight into the user’s environment, what their needs are, what their goals are for whatever your product might be. The results of the research from contextual inquiry can provide information for the requirements for the product. Contextual inquiry is a great tool to use before you even begin product design or working with the current product to find out what is working and what is not working or even the competitor’s product in a contextual inquiry so you can see where your opportunities are, where the pain points are with the current product, and the user’s experience with it and then gather requirements to build a new product.
You may be wondering why I haven’t mentioned focus groups yet, and the answer is yes, focus groups are definitely in our UX research toolkit. Many people who are prospective new clients come to us saying, “We’d like to engage with you to do some focus groups,” and we respond by saying, “Tell us what your goals are in terms of your research.” Oftentimes people come with the idea that they’re going to do focus groups because that’s all they know and of course they’re extremely popular in market research, so we explore what makes sense in terms of the best tool or combination of tools that work for the prospective client, so it may or may not be focus groups.
When we conduct focus groups, however, we like to do user task-based focus groups. We don’t just bring people in to talk about something, but we get them to engage with the interface or the product or paper or anything that reflects wherever the product is in development and get reaction to hands-on experience with it. This is because they’re speaking from what they have just learned or understood rather than from what they think they might like to do. So, there is a place for focus groups, but, like any other type of research plan, it really depends on what your goals are for the research about user experience and which particular tool or combination of tools is going to be the most useful and productive for you.
So, the bottom line is: Every discussion of a user research project begins with what you want to learn and then matching those objectives to the right tools that fit your budget. Every situation is different and matching the right tools to meet the needs of the project and the budget is what helps determine the best single tool or combination of tools for every project.
Customer Experience vs User Experience (CX vs UX)
If a company has customer experience in place, and most companies do, the question is whether UX is a part of the CX research and understanding of the user’s experience in the entire customer journey.
This short video takes a look at what UX contributes to the CX understanding of customers in the customer journey. If you think of customer experience as a pie, for example, where is UX in the pie? Is it a piece of the pie, or is it perhaps not mixed into or baked into the ingredients of customer experience?
What is CX?
CX (Customer Experience) focuses on brand loyalty and likelihood to recommend.
If user experience is not baked into the ingredients of customer experience, is it because perhaps the assumption is that the CX research or the CX team already knows about UX?
Or because perhaps interaction designers have some UX background or because CX covers UX in a way that UX isn’t required.
I would challenge that assumption, if that’s the assumption that’s being made, because UX (User Experience) focuses on the literal experience of your users. It focuses on users engaged with your interface, engaged with your product, engaged with your company in a way that is live, real, in person, and in the moment.
What is UX?
UX (User Experience) focuses on the immediacy of user interaction with your product.
That sort of research is essential to understanding how your users feel about their experience in the whole picture. So if UX is not a part of CX because it’s either a given or it’s assumed to be there when perhaps it’s not, then it might be good to ask whether the addition of a UX person or even a UX team to contribute to the research of the whole customer journey would not provide insights into user experience that inform the total customer experience and that make your users more satisfied , more confident, and happier all around.