How many customers should you test with?

Jakob Nielsen famously wrote about "Why you only need to test with 5 users", and Steve Krug gave a similar variation in his book "Don't make me think". Jeff Sauro digs deeper. He's really into his numbers, so if statistics and probability are a turn off for you, just skip to the conclusion.


The main thing to take away from Jeff's analysis, and his summaries of other usability researchers' work is that while 5 users is a sound basis for basic level user testing, you're not going to catch everything. There is a risk you may even miss some of the big problems.

But what can be said is that testing with 5 users, you are more likely to find most of the significant than not.

And of course, iterative testing with small groups of users is what really brings the benefits.

So while all this might sound a little disheartening to the enthusiastic amateur, remember - testing with even one user is better than no testing at all.

Articles & resources from Jeff Sauro (measuringusability.com)

How many customers should you observe? - Jeff outlines a user observation technique he calls 'follow-home' and provides a calculator that can tell you things like: "To observe 80% of the behaviours at least 20% of my customers exhibit, I need to test with 8"

Why you only need to test with five users (explained) - a discussion of the key points around Jakob Nielsen's famous article, including some nice little interactive tools to illustrate the main statistical points. (The maths teacher in me coming out a bit here!)

A Brief History of the Magic Number 5 in Usability Testing - it seems it wasn't just Nielsen talking about the magic number 5!

And Jakob Nielsen's famous article:
Why You Only Need to Test with 5 Users