Generative vs. Evaluative User Research: Where They Go Wrong and How to Fix It
TL;DR: A recommendation that asks a leader to shift their strategic direction is far easier to dismiss than one that refines a feature the team is already on board with and committed to building. The key is to get in the room early, shape the question, and make recommendations impossible to ignore.
Years ago, I ran nine studies for a founder. They varied widely, but one learning stood out: a whole group of users couldn’t adopt his product without specific features he’d chosen not to build. But the research wasn’t asking him to adjust a couple features. It was asking him to change his strategic vision. At the time, he told me it didn’t fit his direction, so they didn’t implement the findings. But over the next few years (after experiencing the consequences of inaction), most of those recommendations did end up getting shipped.
Being right, it turns out, is not the same as making an impact at the right moment. That gap is what this article is about. In addition to my own experiences, I’ve also gathered input from five people who regularly receive and act on research to share what happens on their side when user research fails to get implemented. Additional sourcing details below.
Two Kinds of Research and How They Fail
User research does two different jobs. Sometimes it generates. It surfaces a direction, reframes the problem, and tells you that you might be focusing on the wrong thing. Other times it evaluates. It assesses something that exists and may recommend improvements. Many studies do both, but an individual recommendation usually leans one way or the other: improve the current approach, or consider rethinking the overall direction.
These two kinds of research recommendations fail in different ways, and when the generative kind fails, the consequences tend to be worse.
An evaluative recommendation might ask a team to change its backlog or a design, or implement a fix. In contrast, a generative recommendation may ask a leader to reconsider the strategy, market, or bet they have already chosen and defended.
One of my sources, Robin Reynolds, a VP of Product, elaborated on the tension that generative recommendations can create:
„The goal of research is not [to find] the solution. The goal of research is to pick the right problems, and then we develop a solution. When you come in with the solution in mind already, you’ve closed the door on being open to hearing what we learn.“
When leadership has already settled on a solution, generative research feels less like a contribution and more like a threat. And the failures that follow occur because the door to research making an impact was closed too soon, or never open to begin with.
Why Generative Research Fails Harder
The biggest failures I’ve heard of all came from ignoring generative research.
For example, a study that Robin recalled, which concluded that the strategy itself was wrong:
„We were trying to force the research to validate a direction that we wanted to go, and what we actually learned was that’s not the right direction. We shouldn’t go that way.“
The company went ahead with the direction anyway, and later shut down. That’s what can happen when generative research is right about the direction of strategic decisions, but gets overruled by executive opinion.
In another big, foundational project that involved rethinking a core workflow, the research findings called for a full year of foundational work as the solution. According to Robin, leadership balked at the timeline. The generative recommendation was sound, but because leadership didn’t want to slow down and do the foundational work first, the push for speed won, and the foundational work never happened.
There’s a clear pattern here: evaluative research can impact your team’s roadmap and backlog. But roadmaps and backlogs get reshuffled all the time. In contrast, generative research threatens a decision or direction that someone in leadership has already made and defended. These are the kinds of decisions that people are going to fight hard to protect.
That’s why the best fix for generative research isn’t to try and repackage the suggestion or even create a more compelling case for it. It’s to get research involved before the decision has been made and defended in the first place.
The Most Consistent Advice: Go Upstream
I asked each person I interviewed the same question: What can a researcher do (before handing anything off) to get a recommendation implemented?
They all gave the same answer: go upstream, not only to win agreement, but to help set the direction while it’s still being set.
Diane Chang, a senior product manager, recommended going for an up-front discussion with the product team about what they want research to help them with so that research can be involved from the very beginning of the decision-making process.
Robin proposed going straight to the “right people” to ensure that the ones ultimately making the final decisions are not only aware of the research plan, but approve it so that there will be no chance of them saying something like, “well, you didn’t do it the way I would have, therefore we’re not taking the recommendation.”
Deb Davis, a design director, suggested doing discovery on the executives in order to better understand the business and learn “where their minds are” to ensure that the research is built around something that is meaningful to them.
The pattern across all three pieces of advice is that the fate of a recommendation is mostly decided at the planning stage. You need to know:
Who’s in the room when the questions get set.
Whether the study is anchored to a decision someone owns, and who that owner is.
Whether the research is generating the direction, or merely auditing a direction someone else already chose.
Get there early enough, and your research stops becoming an argument that people who already have a strong direction in mind need to accept, reject, or shelve. It becomes part of how the plan itself gets made, and people are less likely to argue with a plan they helped write.
Four Ways Evaluative Research Can Fall Apart
In addition to the challenges inherent in generative research that can cause it to fail, there are also four ways that evaluative research can fall apart that you should watch out for.
1. It Loses to a More Valuable Priority
This was the most common response, and everyone mentioned some variation of it. Femke van Schoonhoven, a product design manager, laid it out based on her experience:
„Design and product jointly vet it against everything else on the roadmap and against team capacity. If it survives that filter, it moves into scoping; if not, it sits in the backlog, sometimes indefinitely. It’s not that anyone disagrees with the finding. The expected impact doesn’t clear the bar relative to other work fighting for the same resources.“
Sometimes, a research recommendation doesn’t actually get rejected. It just dies a slow death as it keeps getting pushed down the priority list week after week.
If your suggestion keeps losing out to other priorities, but you’re confident it deserves attention, try building a stronger case for it by highlighting the business impact to convince the leadership team that delaying it has a real cost.
2. The Implementation Cost Is Bigger Than It Seems
Sometimes, “we can’t just do that” is simply the truth because of technical limitations.
Lou Giacalone, a developer and serial entrepreneur, explained why coding can sometimes stand in the way of good suggestions:
„Code is brittle. There are very few things in the code that are isolated. They’re all talking through other pieces of the code. Drop [a new line of code] in the pool, you’re gonna get this wave, and this wave goes out, and it touches everything.“
Developers and others who work closely with legacy systems tend to spot these ripple effects early. PMs, designers, and researchers usually notice them later when they show up as a high estimate, a delay, or a roadmap trade-off.
If the blocker is technical, try asking the developers what a feasible first step would look like, or agree on a timeline that gives them more room to plan the changes properly.
3. No One Actually Owns the Decision
Researchers rarely own the final decision. Research recommendations get passed on, and it’s not uncommon for them to catch teams by surprise or end up with someone who’s unable to move them forward. This can result in final decisions simply not being made.
Diane described this exact scenario in which her research suggestions ended up at a dead end. She shared that she’d provide the evidence to the team owning the product area, but wouldn't necessarily have a direct role in their prioritization. As a result, the recommendation didn’t get rejected, but ended up stuck with someone who didn’t have the power to act on it.
Before the study starts, find out who owns the decision your research will inform and get their commitment to act on it. Name that person in the plan document so accountability is explicit. An insight or recommendation the decision-owner helped scope is far more likely to be used.
4. It Never Got Specific Enough to Act On
This one’s actually on us, the people conducting the research, which means this is something we can fix immediately.
At face value, it might look like an execution problem, but it’s really a failure in setting direction.
In our exchange, Diane recalled a market analysis whose findings were too high-level and did not break down potential users into segments, so the recommendations weren’t actionable. The research was meant to guide the team’s next steps, but with nothing concrete to work with, it ended up being ignored.
Femke shared some blunt advice on how to fix this: come with recommendations already prioritized rather than a flat list of findings. Otherwise, someone with less context than you will decide what matters, or ignore the research altogether.
Conclusion
Research recommendations rarely go unimplemented because nobody believed them. More often, they lose out behind closed doors, in trade-offs you weren’t aware of: a priority you didn’t know they were competing with, a technical cost you couldn’t estimate, a handoff with no timeline attached, or, hardest of all, a request for someone to change their mind after they’d already made it up.
That’s not something that can be fixed with a more polished deck. The leverage tends to live at the two ends of the stream.
Upfront: get in the room while the direction is still forming, so research helps decide what’s worth pursuing instead of showing up late to critique a decision that’s already been made.
At the back: make the recommendation something people can see, react to, and act on, not just an argument they can dismiss.
Whether your research is generative or evaluative, it pays to get in the room while the questions are still being set. That’s your chance to ask the person who owns the decision what they’re actually trying to decide, so the research can be designed to inform that decision rather than arriving after it’s already been made.
Pressure-test the plan with that person and with the people who will build on your research, like designers, engineers, and PMs. That way you can learn what the potential blockers might be while there’s still time to change course.
Then when the research is done, and you’re ready to write the recommendation, make sure it’s specific enough that it can be acted on without you in the room to explain it.
A Note on Sourcing
I gathered input from five people who regularly receive and act on research: two product leaders, two design leaders, and one developer and serial entrepreneur. Three participated in conversations, and two responded to the same questions by email. All five work inside tech product organizations, from a 10-person startup to a top-10 technology company, not in packaged goods, pharma, or other sectors.
Michele Ronsen is a researcher, educator, and builder, and the founder of Curiosity Tank. With 20+ years of experience, she specializes in UX strategy, user research, service design, and innovation. She has taught more than 7,000 hours of design and research programs, and she advises Fortune 500s, startups, and frontier AI organizations, including Cohere, Stripe, Slack, and Amazon.
