The right way to ask the right research questions

Feedback from customers is a vital input for product development. But if you want to get the most out of product research, you’ve got to ask the right questions.

A good way to go about this is to start at the end (Google Ventures have some pretty solid advice on how you can do that). Think about what you want the outcome of the research to be. Do you want to understand what customers are using your product for? Or are you trying to understand how successfully it meets their needs? When you define the questions you’re really trying to answer, you can work backwards and craft the right research to get you there.

Here’s a short guide to some questions that you can ask to achieve different research goals.

Evaluating the success of a feature

If you’ve shipped a feature and you want to understand whether it’s doing well or not, it’s good to ask the people who are actively using it. By doing this, you’ll be able to reveal any underlying issues users might be having, while also getting great feedback on how you can improve and iterate on what you’ve built. Remember to keep it simple and straightforward by asking a sequence of questions like these:

1. “What works well about X?”
2. “What’s confusing, if anything, about it?”
3. “What could we improve?”

This will allow the user to think logically about what they like/dislike about the feature. Another important factor here is that by asking for good and bad points separately, the user won’t feel obligated to only give positive answers as you’ve remained balanced in your questioning.

Understanding user intent

So your analytics shows you customers are heavily and repeatedly using a particular feature. Great. But do you know what they’re actually using it for? They may not all be using the feature for the same reason. Digging deeper on their goals could shed light on your customers unmet needs.

If you want to understand the intention behind usage, it’s smart to ask them a question about the feature just after they’ve used it. This means the feedback you receive is likely to be much more accurate as you’re not asking the user to recall an experience from a long time ago. Human memories of product interactions can decay fast. Some questions to ask in this scenario would be:

• “When you used X just now, what were you trying to do?”
• “The last time you used X, can you tell us more about what were you trying to do?”

Gathering initial perceptions of a feature

First impressions of new (or improved) features can help uncover whether users understand your product/feature quickly and easily. It helps identify common misunderstandings that might prevent others from using the feature (e.g. the feature is hidden under a menu item that sounds nothing like the actual feature it links to).

This early-stage feedback can inform any further design decisions that you’ll make as you iterate on the feature. If you want to get valuable feedback on a new product/feature, it’s good to conduct research right after a release but the timing can be adjusted based on usage. Keep your questions broad and open-ended so the user doesn’t feel they need to hone in on a certain area (such as the aesthetics of the design):

• “What has your impression of [feature X] been so far?”
• “You’ve been using [feature X] for a few days now, what would you improve about it (if anything)?”

Gathering more considered feedback on a new feature

If you want more considered feedback on usage, then it’s better to wait, or run a second wave of research after the user has had a chance to use this feature and adjust to the changes it may have brought to their workflow. The timing of this adjustment period will depend on how frequently people use your product. If people use your product on a daily basis, you could even consider starting research after a few weeks. But if customers tend to use it heavily once a month e.g. an invoicing app, then problems likely won’t manifest themselves for a few months, which equates to a few usage cycles.

Pro tip: If you redesign a key feature, it’s likely you will trigger some change aversion amongst your users.  For a short period of time, users shift towards either loving or hating the change. In the research world we say change aversion typically manifests itself by a sudden shift to a bimodal distribution after launch. Usually this is followed over time by a return towards something similar to the normal distribution of the pre-launch period.

Good questions to ask here:

• “What has your impression of [feature X] been since [change was introduced]?”
• “How has [feature X] changed the way in which you work (if at all)?”

Investigating users’ mental model of your product

To really understand your users, you need to understand the mental model they have of your product. This is particularly important when you’re building something totally new that has few parallels with other products in the market or if you’re redesigning core pieces of your product. To gain insight into your users’ comprehension of your product, ask questions that will immerse you in the mindset of your customers:

• “Explain in a couple of sentences what you think the product does”
• “If you were to explain [product/feature X] to a friend, how would you describe it?”

Figuring out how users intuitively expect your product to work will help you fine-tune it, if necessary, to the mental model of your users (thus making it more intuitive for them to use). If you think there are features that might be unintuitive, you can evaluate this by asking open ended questions like:

• “What do you think the difference is between [feature X] and [feature Y]?”
• “What do you think [feature Z] is and what is it useful for?”

These types of questions will give you critical insights into the way your users think about how your product works, and will ultimately help you build great software.

Once you’ve gathered your feedback, you shouldn’t stop there. Some next step tips include:

Don’t be afraid to dig deeper

When sifting through a big chunk of responses to a product research question, pick out any pieces of information that need to be clarified and follow up with the customer for further information. Consider other types of user research you could run to learn more and try to iron out any statements that don’t have a clear rationale. If you ask “why?” enough, you’ll get to the root cause of why a user loves/hates/is indifferent on a design or feature. As Des mentioned before in his “5 mistakes we all make with product feedback” post:

“Treat every clustering of feedback that you see as a hypothesis, and then don’t build it, verify it. Once you verify that the pain is real, the next step is never “build the requested solution”, you have to go deeper…”

Actionable insights → awesome products

As long as you’re asking the right questions, you’re going to get valuable insights that you can act upon. Following these tips will lead to better insights for your company, whether or not you’re using Intercom. Ultimately the answers you get from users can help you see your product in a completely different light and help you build awesome products your customers love using every day.