Prior experience affects usability scores

Jeff Sauro digs into his back catalogue of usability testing data to analyse whether participants with prior experience of a website or application respond differently to those with no experience.

Are attitudes about usability constant? If we think something is unusable today, will we feel the same next week, next month, next year?

In my opinion  I think it depends on the service. If we find usability to be bad and we have a viable alternative maybe we won't use it again. So returning to it later, yes it's still poor. But if we don't have an alternative (Facebook springs to mind, and a few applications at work) we just get on with it. The incentive to perservere or the lack of an alternative means we learn to overcome (or tolerate) the bad usability. Long term, it's almost inevitable that this affects our perception. We forget some of the bad experience we encountered at the beginning.

A few comments from Jeff:

On average I found that repeat users rated the websites as 11% more usable than first-time users. Generating a confidence interval around the difference tells us we can be 95% confident that repeat users think the website is between 6% and 15% more usable.


When gathering ...usability data it's important to track the user's prior experience with a product or website... Researchers are concerned that the new system gets penalized because it isn't as familiar to the users even though it has bug fixes and simpler designs. They worry that improvements in usability might be masked by the effects of experience.


This data suggests that there is reason to be concerned. Even differences as large as 10% may be caused by differences in exposure.

I've certainly experiences similar in anecdotal comments when working with university colleagues on website and applications developments. "I used to know where x was on the old website, but now..."


Does prior experience affect perceptions of usability? - article by Jeff Sauro for measuringusability.com