This feature reminded me of my own mistakes when I started learning user research to inform product decisions. Poor recruiting, unfocused questions, and handing off multi-page research reports no had time for, I cringe thinking about the basic mistakes I made.

Thankfully I’ve improved my ability to use interviews to inform both the usability and usefulness of product development through lots (and lots) of trial and error. Here are a few things I’ve learned about using research interviews to help guide design during my time at Made by Many:

or

These are completely valid questions that we hear often.

My answer is Yes, No, and a question. Because the underlying question to those concerns is why is qualitative research the best tool for the question we’re trying to answer? Because in many cases, in-depth interviews are not the right tool for the right job.

Understanding how to understand research starts with knowing its misperceptions.

Here a few common examples of misusing qualitative research:

Statistically significant: Statistical significance is knowing the results from flipping a coin 50 times will be similar to flipping a coin 5,000 times. Or if you poll a sample size of Ohio voters in a presidential election you can measure how the entire state will vote.

Market Research: This type of research is based on people perceptions and emotions to help you make people want something rather than making something people want.

Determining purchasing intent: Directly asking people “would you pay for this?” and then returning to your team with “but six people said they would definitely pay for this!” This is an unreliable way to determine purchasing intent. Not that those people are intentionally lying, it’s just an unreliable way to be sure who’s behavior will actually match their stated answers.

Asking people what they want: Which we mentioned above and also stated nicely by a Henry Ford quote that he didn’t actually say.

Qualitative research matters because there simply aren’t numbers for everything. At its best, in-depth qualitative research shines light on what numbers can’t measure.

Data will inform you that something isn’t working but not necessarily why or how to fix it. It provides answers when you already know what you’re asking but ‘why?’ is difficult to answer with only numbers.

For example, data can tell you that 70% of women voted for one election candidate over the other. But why? Is it because they didn’t trust the other candidate? Or is it because that they never actually heard about him or her? Or maybe it’s because they didn’t like either candidate but disliked one much more. It’s hard to tell just by looking at numbers.

Good qualitative research explores the problem before the solution. It helps us understand people’s needs and usability issues, showing us what problems to solve. It also gives us empathy and a shared understanding with your team. Ultimately, a solution can only be as good as the depth of understanding the problem.

Poor questions will tell you what you want to hear rather than getting real insight into people’s needs.

I wish I knew who to credit with the phrase ‘the first rule in asking questions is that we don’t act on people’s requests, we act on their behalf’. Because that’s our mindset for conducting research. It’s our job as a product team to solve those problems rather than ask for solutions.

Here are a few other examples of questions to avoid in interviews:

Good interviews reveal how people intuitively solve the problems they may not even realize they have. We focus our questions on actual past behaviors and the reasons why they behaved that way. Here’s one sample framework for uncovering what people do and the underlying reasons why:

Well conducted research will answer some basic questions no matter what you’re working on. Questions like ‘What are people trying to accomplish?, Do their goals change? What tasks do they need to achieve goals?’ amongst many others listed below:

While my primary goal is to use interviews to fill the grid above, my ears perk up when I sense that people are describing something they do but no one’s ever asked about. For example, I was conducting an in-home interview when we discovered that as part of her dental routine, the woman puts down her toothbrush and tries to sense with her finger which of her teeth still ‘feel dirty’ before re-brushing those areas. Her behavior showed us that she wanted some sort instant feedback on how well she’s brushing her teeth in a way her words didn’t articulate.

Another example is when we were interviewing teenage athletes about buying speciality athletic gear (vague, I know... but we give away too much about the client by mentioning the sport). This is an example of how far a kid was willing to go get the right gear in his size.

People are already solving their problems somehow. Their workarounds are so obvious the people don’t even realize they use them. We know how much a problem matters to someone by how far they’re willing to go to solve it. Done correctly, interviews can show us what numbers can’t measure and reveal workarounds that people don’t even realize they have.

Which hopefully teaches us everything we need to keep our stores cluttered.