Plan to learn — on well designed research plans

Well designed research plans in qualitative user research help to successfully learn about users

I love user research. I’m fascinated with talking to people and observing them while they do their stuff. I’m fascinated so much, sometimes I almost forgot why I’m there: to learn about a specific question we set out to answer. Research plans remind me to focus on the important questions.

Heroes need plans

Research plans

User research plans come in all kinds of flavor. They vary from a simple script containing a few questions to ask, to a strategy on which methods to use and how the insights build up from the initial questions to the final report. I like to refer research plan as something in the middle, driving one qualitative study with a handful of participants.

There are a number of excellent resources on planning simple studies, but you don’t even need that many points, especially if you are working with continuous discovery.

The perfect research plan only needs five elements:

• Goal: why are we doing this research, what are we hoping to get out of it.
• Research questions: what are specific questions we are interested in.
• Method: what specific method we want to use to learn about the questions.
• Participants: what’s the user profile we are looking for.
• Protocol: how we want to conduct the session.

You would want to also add a specific title, like “2018–02–29 Onboarding user interview” so you can differentiate from your other plans, and your name in case somebody has questions on the details later on.

Goal

The study’s goal is sometimes hard to set, as people get so excited that they would jump straight to writing up their questions. Stepping back and thinking about the purpose is important as it provides you the lens for filtering questions and deciding on participants and research method. If you don’t define your goal, what you want to learn and achieve with this study, you’ll never know for sure if you succeeded.

Defining the goal should be done by collaborating with the stakeholders (for example product managers). Such discussions also help to uncover assumptions, and risky areas in a project that would need further investigation with other methods.

A great goal shouldn’t be more, than 2–3 sentences and should clearly state what is the experience, workflow or task you wish to learn about, what aspects are important, and also what you hope to achieve with this knowledge.

Research questions

A list of questions on what you intend to learn with the research. I like to set this list as wide as possible, as long as the questions are within the scope defined by the goal. Sometimes not every questions is possible within one study, or even important to answer. But users tend to have the ability to tell you things you didn’t expect, so thinking about what else could be interesting will help you recognize a piece of interesting insight from a word the user randomly utters.

Research questions can be quickly brainstormed with project participants, after making sure everyone has a shared understanding on the goal. I usually find not only product managers or designers, but also engineers have excellent questions.

A great list of research questions are specific, not overly general and focus on new information to learn about the users (as opposed to learn something we can get from market research or analytics).

Method

Once you have your goal and your research questions, you can decide which method to use. That is, which method will be the most suitable to answer your questions. For generative research the most common answer is some type of user interview, but there are plenty of possibilities to choose from.

Describing your method in a few sentences also helps team members and stakeholders understand how you will work. And maybe how you expect others to participate in the sessions, or how will they learn about the results.

Participants

With your goal, research questions and method set the user profile you want to research with. This can be even some demographics or a persona description, but the best is to describe it with behaviours, so “users who do x”. Maybe even some hard metrics, like “shopping 4 times per week”. Specific info here will help in setting up the recruitment screener and make sure you get the right type of participants. These specifics can be written based on the goal.

Choosing a somehow restricted user profile has an other benefit. If you cannot find participants for your research it may very well be that the original problem you set out to research is not really a problem at all, so gives an early validation on your product or feature idea.

Protocol

I see protocol sometimes confused with the research questions. In essence research questions are what you want to learn, while protocol is how you will learn it. Protocol is a guide on how to facilitate the user session, especially in case of a user interview or similar methods where you interact with the user directly.

Why you need a protocol, if you already have your research questions? Unfortunately just plainly asking the users what you want to learn will probably lead to biased information, in some cases even falsehoods. For example an always interesting question is how much users are willing to pay for a product and if they would use it. If you ask this from the users though, it’s unlikely you will get an informative answer, as people are not really good at predicting their own future behaviour. So protocol should contain most of you research questions in a well formulated way.

A good protocol does a few other things besides containing a set of well-formed questions:

• Gives a logical structure to your session.
• Defines time allocation (for example if you have limited time for an interview).
• Has an exact, word-by-word description what you need to tell in the beginning of the session (like you will do a recording), and at the end of the session (like how you will handle the incentive).
• Helps not only in the session, but also in the note taking.

Protocol doesn’t need to be followed linearly, it’s a guide you can lean on while you facilitate the session.

The rest

There may be some additional elements in a research plan, however I find these are optional and not needed in all studies.

• Hypothesis: Assumptions, hypotheses, validation brought much rigour into user research, and shaped product discussions into a more well thought-out direction. But at the same time with qualitative research it’s though to talk about validation, as you collect a limited number of facts. Still, talking about hypotheses might be useful as it opens up discussions on what we think we know and might bring in new questions.
• Prototype: If you have something to show to the user to discuss about, it’s worth preparing and linking the proto upfront, so you can find it.
• Time frame: Qualitative research might not have a definite end, even if patterns already emerge you learn new details all the time for a specific set of goal + questions. It’s probably worth to time box the efforts, and instead of doing one long study, stop after a few weeks, do a round of synthesis and plan an other study based on what you’ve learned.
• Recruitment: Details, such as the screener or where the advertisement was posted may be worth noting, as these might uncover how data can be possibly biased.
• Budget: Depending on how you allocate your budget, recruitment and incentive fees other costs like renting a facility might be important to note.

What you cannot plan for

One important thing to know about plans: they tend to not behave exactly how we intended them to behave during usage. Or, as Helmuth von Moltke the Elder put it:

No battle plan ever survives contact with the enemy.

Though research plans is not about contacting the enemy, it still concerns humans, so a number of things can go wrong.

• Goal: You didn’t learn what you wanted to learn. One cause could be that the questions, the participants, the method or the protocol didn’t remain in the scope the goal defined. It can also happen the initial goal didn’t make sense, try to learn about it in some other way, like desk research.
• Research questions: The questions were not the one you were interested in after all. This can especially happen at the beginning of the projects, when the even the scope is unclear, and as you begin to talk to users, they will shift your focus into a different direction.
• Method: Some methods may not work for certain contexts, which is difficult to predict if this is a new topic for the researcher. For example in a certain domain users are way less likely to share or willing to talk, so some other method should be chosen.
• Participants: Users arrive perfectly fitting the profile, but they are not the ones you were looking for. Maybe the profile was too strict or loose. Maybe the type of user you had in mind does not exist. Maybe the profile was not clear, and it resulted in a misunderstanding with the recruiter.
• Protocol: The users don’t answer your questions, or give biased answers (which is by the way difficult to detect), or are puzzled by your questions. Protocol questions may need to be reformulated several times until you get to a version, which help you to learn actionable insights.

All in all, research plans, especially for qualitative research shouldn’t be set in stone. You should learn from each session you conduct, and iterate on your plan.

Planning to learn

A well crafted research plan guides you during sessions and helps in learning about the user, but don’t forget it just guides. Sometimes the best insights are gathered when you deviate from your plan.