UX Research Sample Size Considerations

Before our user experience research team kicks off any study, we must justify our participant sample size. Clients and colleagues will want to feel confident that they can trust a study’s recommendations, and that is hard to do without faith in the study’s sample population and size. It is beyond, “Did you talk to the right people?” It is also, “Did you talk to the right number of people?”

Quantitative research methods (e.g. surveys) have effective statistical techniques for determining a sample size based on the overall size of the population being studied and the level of confidence desired in the results. Qualitative research methods (e.g. interviews) currently have no similar commonly accepted technique. Yet, there are still steps we can and do take to collect and analyze the right amount of data.

Scope

We begin by considering what we are trying to accomplish. This is an important factor when deciding sample size. Are we looking to design something from scratch? Or, are we looking to identify small wins to improve a client’s digital product or service’s usability?

We would maximize the number of research participants if we were looking to inform the creation of a new experience. We can include fewer participants if we are trying to identify potential difficulties in, for example, the checkout workflow of an application.

User Characteristics

We will increase our sample size as the diversity of our population increases. We want multiple representatives of each persona or user type for which we are designing. I typically recommend a minimum of three participants per user type. This allows a deeper exploration of the experience each user type might have.

For example, if we are designing an application that allows manufacturing companies to input and track the ordering and shipping of supplies from warehouses to factories, we would want to interview many people involved in this process: warehouse floor workers, office staff, procurement staff from warehouse and factory, managers and more. If we were simply looking to redesign the interface of the read-only tracking function of this app, we might only need to interview people who look at the tracking page of the application: warehouse and factory office workers.

As researchers, we want to provide our clients and colleagues with greater clarity into how we determine sample sizes for any research engagement we undertake. Scope and user characteristics are always two important factors we consider when determining a study’s sample size.

Illustration by Brandon Sax