Five? Why do you recommend five testers?
We actually don’t, Jakob Nielsen does. Jakob is a famous usability expert. In fact, prestigious publications call him the following wonderful things:
To read his full biography, visit his website here.
Jakob wrote the defining article on how many usability testers you should use to test your website. To read the full version, click here. In it, he states:
He goes on to state:
The most striking truth of the curve is that zero users give zero insights.
As soon as you collect data from a single test user, your insights shoot up and you have already learned almost a third of all there is to know about the usability of the design. The difference between zero and even a little bit of data is astounding.
When you test the second user, you will discover that this person does some of the same things as the first user, so there is some overlap in what you learn. People are definitely different, so there will also be something new that the second user does that you did not observe with the first user. So the second user adds some amount of new insight, but not nearly as much as the first user did.
The third user will do many things that you already observed with the first user or with the second user and even some things that you have already seen twice. Plus, of course, the third user will generate a small amount of new data, even if not as much as the first and the second user did.
As you add more and more users, you learn less and less because you will keep seeing the same things again and again. There is no real need to keep observing the same thing multiple times, and you will be very motivated to go back to the drawing board and redesign the site to eliminate the usability problems.
After the fifth user, you are wasting your time by observing the same findings repeatedly but not learning much new.
Jakob states that testing with 5 users uncovers 85% of all usability problems with your website, and that by testing with 15 users, you can uncover nearly 100% of all usability problems. So why not test with 15 users? Jakob goes on to say:
So why do I recommend testing with a much smaller number of users?
The main reason is that it is better to distribute your budget for user testing across many small tests instead of blowing everything on a single, elaborate study. Let us say that you do have the funding to recruit 15 representative customers and have them test your design. Great. Spend this budget on three tests with 5 users each!
You want to run multiple tests because the real goal of usability engineering is to improve the design and not just to document its weaknesses. After the first study with 5 users has found 85% of the usability problems, you will want to fix these problems in a redesign.
After creating the new design, you need to test again. Even though I said that the redesign should “fix” the problems found in the first study, the truth is that you think that the new design overcomes the problems. But since nobody can design the perfect user interface, there is no guarantee that the new design does in fact fix the problems. A second test will discover whether the fixes worked or whether they didn’t. Also, in introducing a new design, there is always the risk of introducing a new usability problem, even if the old one did get fixed.
Also, the second test with 5 users will discover most of the remaining 15% of the original usability problems that were not found in the first test. (There will still be 2% of the original problems left – they will have to wait until the third test to be identified.)
So there you have it. The top web usability expert in the field recommends you usability test your website with 5 users, with three rounds of usability testing. We strongly agree with Jakob, which is why we made running usability tests so affordable, and so simple for you to create.
If testing with 5 users is a bit much for you, why not try testing with one tester? To make it risk free for you, we’ll give your first test free.