Choosing the Right AI UX Research Tool: A Strategic Guide for UX Teams

This article was written by Dr. Llewyn Paine, an external UX expert offering her perspective on the key factors user researchers should consider when evaluating and choosing an AI UX research tool. Her views, informed by her extensive experience in the field, shed light on the factors that truly support effective analysis and trustworthy results.

Deciding on your AI research tool stack isn’t just an operational choice. It’s a UX strategy decision. When selecting an AI UX research tool, it's important to consider how it will integrate with your larger design process to ensure seamless collaboration and actionable insights.

Especially as more non-researchers opt into conducting UX research themselves, your tools and practices will set org-wide standards for how everyone conducts research with AI.

To navigate this responsibly, research leaders should examine four critical questions:

1. How do AI tools actually work for qualitative data analysis?

2. How will your AI research tool choices shape business decisions?

3. What’s the likelihood of AI features producing errors?

4. What kind of research culture do you want to foster?

This article focuses on AI for qualitative research analysis. Many of the top AI tools for UX research (in terms of popularity) are analysis tools, which makes them of particular strategic importance. Other tools and emerging research applications, such as AI moderation or synthetic users, come with their own risks and are outside the scope of this article.

Whether you’re using ChatGPT, your company’s internal chatbot, or AI features in a UX research tool like Condens, much AI research functionality today is powered by large language models (LLMs).

These models don’t analyze user research data the way humans do. They predict plausible responses based on patterns from their training data. So even when we talk about “reasoning models,” there’s no actual “thought” involved (at least as humans mean it).

Because building custom LLMs is costly and complex, toolmakers typically rely on commercial models such as GPT-4o or Claude Sonnet 3.5. This means:

• Most platforms have similar core capabilities

• Most share the same underlying limitations

Human qualitative analysis is a sensemaking process. Analyzing qualitative data by reading text, pulling out meaningful quotes, building conceptual frameworks to draw conclusions, and testing them against counterexamples and edge cases is central to qualitative research. It relies on in-depth, human-centered investigation methods to uncover rich insights into user behaviors, motivations, and experiences. Approaches that are distinct from AI-driven methods.

LLMs are incapable of this type of sensemaking. Instead, they look for patterns that match their training data. This makes them good at generating results that have a generic sort of plausibility, but they can’t draw new connections that aren’t already represented in the examples they've seen before. They may also generate stereotyped results that sound reasonable, but are not representative of your actual data (for example: I’ve had LLMs claim to discover generic usability issues in user testing transcripts they couldn’t read).

This is not to say that data analysis by LLMs is worthless. Quite the contrary! Semantic, structural analyses of text can be a powerful way to explore qualitative data. But we need to recognize this as its own distinct method and call out the differences when using it in user research analysis.

According to research from OpenAI, “[LLM] accuracy will never reach 100%” because errors and hallucinations are fundamental to how LLMs are trained (changing this would require a complete process overhaul).

To help with this, many UX research platforms have infrastructure that is intended to make it easier to fact-check AI output.

For example, whenever Condens surfaces AI-generated claims, it provides multiple supporting quotes from original sources. These quotes are post-processed outside of the LLM to ensure that they are accurate and link back to the specific participant, research session, and project metadata. This lets UX researchers easily see what participants actually said in the original context.

These types of precautions can help AI-powered tools stay more grounded in user research data, and make it easier to check claims against participants’ actual quotes.

But even the best-engineered UX research platform cannot eradicate all risk of error, so you should definitely fact-check all AI-generated results.

Human UX researchers use standardized techniques to develop accurate, representative, and actionable insights, and also to ensure they don’t overlook unexpected findings. Adding LLMs to the mix introduces a new element into the research process, as well as new types of errors. 

Common errors include:

• LLMs generating themes that sound plausible but aren’t representative of the data

• LLMs generating themes that are superficial

• LLMs fabricating or misattributing supporting quotes as evidence

• LLMs citing irrelevant quotes that (while technically accurate) don’t support their claims

This list comes from many controlled tests, run by dozens of UX researchers worldwide, with each test manually evaluated to observe exactly what types of problems are most likely to occur.

(I ran these tests as a series of workshops. You can find more details in this report.)

It’s clear from these studies that there’s a fair chance of error or superficiality any time you use LLMs in UX research analysis, which introduces a degree of risk.

When using an AI user research tool instead of working directly with the LLM, there are often various safeguards in place to help mitigate some of the issues outlined above.

These questions help you evaluate how well AI research tools handle trust, transparency, and safeguards, before you rely on their outputs.

As a research repository, Condens offers a helpful example of what this looks like in practice. The following overview of their safeguards was shared by a company representative and includes some commentary from me:

• When AI-powered search is used to look up information that isn’t in the repository, Condens explicitly states that the information isn’t available rather than fabricating an answer. This helps limit the occurrence of hallucinations.

• Users can precisely control the scope of raw data that the AI search takes into consideration by using filters in combination with their questions. This is not entirely possible, or at least not as reliable, with LLMs.

• Condens uses an evaluation pipeline to surface only the most relevant quotes (to the extent possible using automated judging criteria), and post-processes supporting quotes to ensure that they are accurate and correctly attributed. [Every LLM I’ve tested fabricates quotes or attributions at least occasionally, so having separate infrastructure to tie quotes back to their source is a significant benefit of using a research platform. –LP]

• Condens optimizes their model selection and prompts for UX research through evaluation on test data. This helps ensure you aren’t introducing avoidable errors into your analysis. [This is something I recommend that UX researchers do for themselves when working with LLMs directly. –LP]

• Condens enforces clear consent and labeling anytime AI-powered features are used, and employs AI only when users have opted in. For example, Condens supports automated tagging with AI-suggested tags rather than auto-tagging everything on its own, keeping the researcher in the drivers seat.

• The stakeholder repository in Condens only shows output (AI or otherwise) that’s been researcher-approved.

Safeguards like these can help to significantly reduce the number of AI hallucinations that make it into generated output and can increase AI reliability to ensure greater consistency.

So for those who don’t have the time or expertise to test LLMs for UX research themselves, it may be safer to trust a toolmaker’s setup. Plus, using the same platform across an organization ensures that everyone stays on the same page when it comes to the models and prompts used.

LLM vs. a user research platform is not the only choice that will impact your research reliability, however. It will also be impacted by the specific AI features your team uses.

AI Clustering features usually appear on a virtual whiteboard, where the UX research tool automatically groups similar notes, quotes, or highlights from qualitative data. This is typically done using semantic similarity or sentiment analysis and can save you time by eliminating the need to sort sticky notes yourself.

Risks:

• Clusters may stay surface-level

• Deeper themes may not emerge

Strengths:

• Highly transparent and traceable

• Able to assign your own labels

• Lower likelihood of fabricated claims

AI Summaries generate short narratives based on a document or group of notes.

Risks:

• Summaries may be superficial and fail to actually highlight the important points

• Potential for subtle hallucinations (for instance, stating that a behavior was “often” observed, when it actually only occurred once)

• Supporting quotes may be irrelevant or insufficient

Strengths:

• Easier to verify when AI tools provide transparent citations

• Can be useful when treated as a starting point for data analysis, and not as a complete user insight

To combat false claims, many AI Summary features offer citations, allowing the researcher to view the supporting quotes. Platforms like Condens may also have dedicated infrastructure to ensure that quotes cited are verbatim and correctly attributed, making verification easier.

So AI Summary features do involve some risk of superficiality and fabrication, but are generally verifiable with a little additional effort.

AI Reports simulate a more in-depth analysis across multiple documents (e.g., reporting themes across multiple participants). This fulfills the AI promise of instant insights, but at a high cost.

Using AI to generate “insights” across multiple documents is riskier than AI Clustering or AI Summaries because:

• When LLMs do more steps of the analysis, they’re more likely to introduce the issues we discussed earlier (e.g., hallucinations, superficiality).

• They’re harder to verify. AI report generation is prone to hallucination. And because results come from more sources and are meant to simulate a more complex analysis, they require more extensive effort to check.

There’s no way to verify an AI-generated report without doing a substantial level of analysis yourself. So even though the tools themselves may warn you to “verify all details,” it’s extremely cumbersome to do in practice. And the temptation to provide AI Reports as instant research findings to stakeholders can be hard to resist, particularly for team members who haven’t been trained on the risks (this is an ongoing issue in legal research).

So while AI-generated UX research reports are in high demand because of their ease of use, research leaders should think twice before adopting them and making this feature available to their team. Especially when considering how they may be used by novices.

Whether you’re using an LLM chatbot or a purpose-built AI UX research tool, errors in AI-powered research analysis are a reality that isn't going away anytime soon. And your AI user research strategy needs to account for this.

Especially when conducting research that can impact critical business decisions, human researchers should still be involved in the user research process every step of the way. Human expertise is essential for validating AI-generated research findings, interpreting nuanced user behaviors, and making informed research decisions that require contextual judgment.

That’s why it’s important to still have access to features in your tools that support human analysis (which actually isn’t possible with some new AI-first UX research tools).

This can also mean setting expectations for how your team will document and report using AI in their research process. I recommend at minimum ensuring that AI results are clearly labeled, and that you establish who’s ultimately accountable for the insights (AI or otherwise) used in any business decisions.

Adopting AI responsibly in your user research process is all about understanding the limits of AI, communicating them effectively, and adopting tools and policies that clearly differentiate between AI-generated insights and human-derived insights.

This may look like:

• Clarifying where AI is helpful and where it introduces too much risk

• Ensuring transparency around when and how AI is used

• Defining who is accountable for the insights behind business decisions

• Using tools that reinforce healthy UX research habits, not shortcuts

By building these practices early, organizations strengthen the quality of their research and elevate the value of skilled UX researchers.