BetaDojo
How many testers do you need?
In 1993 Jakob Nielsen and Thomas Landauer modelled how many usability problems a test finds as you add testers. In their data each tester found, on average, about a third of the problems, and the overlap grows fast — so five testers found roughly 85% of what many more would. That is where the “five users” rule of thumb comes from.
The caveat: a small test is for iterating — find problems, fix them, test again with five new people. It cannot prove how many people like your project, and very different kinds of users need their own five.
Ask what they did, not what they would do. Rob Fitzpatrick’s The Mom Test (2013) puts it simply: people are kind about your idea and bad at predicting themselves. “Would you use this?” gets a polite yes. “When did you last plan your homework, and what did you use?” gets the truth.
How are you feeling?
A case · you decide
Ines and the twelve sticky notes
Ines, 15 — just watched four classmates try the board game she designed for a class project
Ines has twelve sticky notes on her desk. One says the game is 'boring'. Three say different things about the scoring card. Two people laughed at the same moment and wrote 'the trade card!!'. She has one week before the final version is due.
Testers rarely tell you the fix. They tell you where they felt something: confused, bored, delighted. Those moments are the data.
Twelve notes feel like twelve problems. Sorted, they are usually two or three.
What does Ines do first with the twelve notes?
The "confused" pile points at the scoring card. How many changes should she make this week?
The next group of testers went straight through the scoring card and fought over the trade card. Ines kept the notes, sorted, in a folder labelled "round 2".
Brain break
What next? Pick a door
- More like this ShipDojo Is your project ready to ship? PlatformForge Where does the platform’s dollar go? DistrictForge How do the same votes win fewer seats?
- A bit harder TideForge Can the coast grow without the flood? ProvenanceQuest Who should own this object? ExhibitForge What is your show’s big idea?
- Something different SeminarQuest What question opens the seminar? WorldFeels Which feeling has no English word? GrainQuest Can you feed a region for thirty seasons?