Avoiding Pitfalls: 7 Fails in Test User Recruitment for UX Studies

Avoiding Pitfalls: 7 Fails in Test User Recruitment for UX Studies

October 4, 2023

In the fast-paced world of user testing, a smooth recruitment process is essential for the success of your study. Whether you're in tech, design, or any industry relying on user feedback, understanding common recruitment mistakes is crucial. In this article, I will take you behind the scenes to share my insights on the 7 most common mistakes I have encountered, and I'll provide you with practical tips to ensure your test user recruitment process runs smoothly. So, let's dive in and master the art of test user recruitment together!

1. Inaccurate criteria

Let's kick things off with the classic mistake - inaccurate user criteria. Picture this: you need test users who plan to renovate their homes soon. Be crystal clear in your criteria; otherwise, you risk misinterpretation. This clarity becomes especially paramount when partnering with a test user recruitment platform such as TestingTime. Instead of vague terms like "soon," specify a time frame.

❌ Participant wants to renovate their home soon

✅ Participant wants to renovate their home in the next 3 months

‏‏‎ ‎

❌ Participant owns a premium car

✅ Participant owns a Mercedes-Benz, Porsche, Audi, BMW, Volvo, Lexus or Land Rover

‏‏‎ ‎

❌ Participant with mid-income

✅ Participant’s yearly income between 30k and 50k €

2. Short-notice participant recruitment

Ah, the 'Friday panic' scenario – realizing you need test users for your Monday study. We've all been there, and to steer clear of this mistake, ensure you include test user recruitment in your UX research planning.

❌ Recruit test users on a Friday for a study next Monday.

✅ Start recruiting research participants on a Wednesday for a study on a Monday.

3. Insufficient time slot flexibility

For those conducting studies with a very busy target group, B2B participants, and other hard-to-reach participants, empathize with your target group and provide time slots when they are available.

❌ Providing study time slots between 08:00 - 17:00

✅ Providing time slots from 07:30 - 09:00, 12:00 - 13:00, 18:00 - 21:00, and on weekends

4. Complicated tools for test users

Picture this scenario: You find yourself in a well-appointed meeting room with ten stakeholders, all eagerly awaiting the start of your user test. However, there's a hitch - your user test is running behind schedule. The reason? The test user is struggling with an unfamiliar tool, wrestling with its complexity, or, in a worst-case scenario, encountering technical difficulties that threaten to hinder the entire test. Naturally, your stakeholders' patience is wearing thin, and the situation becomes rather uncomfortable.

To navigate these challenges, here's a strategic pro tip: Conduct a dry run with a colleague from a different department or a friend before your scheduled user test. This proactive step can iron out any wrinkles and ensure a smoother experience when the real test starts, to save you from the hassle of stakeholders drumming their fingers impatiently.

Pro tip 2: TestingTime offers an additional layer of assurance with the option to reserve backup test users. The add-on ensures that even if your primary test user encounters unforeseen obstacles, a backup user is ready and waiting.

5. Screener questions with yes/no answers

Another common mistake when creating your own screener questions is using yes/no answers when asking for specific criteria. Instead, use open-ended questions with various answer possibilities. For more tips on crafting good screener questions, check out the guide on how to write screener questions like a pro.

❌ Do you like chocolate ice-cream?



✅ What's your favourite ice-cream flavour?






Cookies & Cream

None-of the above

I don’t like ice-cream

6. Too many test user criteria

Okay, let's talk about something we all do at some point: going overboard with criteria. You know, like trying to find that one-in-a-million unicorn participant for your research. But hey, sometimes, you don't need a unicorn; a friendly neighborhood horse will do just fine.

So, here's the deal: Keep it simple. Ask yourself, "Do I really, really need those super-specific participants?" Sometimes, your research can groove along just fine with the good general population. Every extra criterion you add is like searching for Waldo in a haystack - it makes life harder than it needs to be.

7. Compensation issues

When it comes to compensation in user testing, it's crucial to strike the right balance between motivating participants and aligning with your research budget. When dealing with hard-to-get participants, fair compensation takes on even greater significance. These individuals may be busy professionals, industry experts, or individuals with unique characteristics that make them challenging to recruit. To effectively engage and retain such participants, consider the following points:

Competitive compensation

Given the scarcity of these participants, it's essential to offer competitive compensation. Research and benchmark what other organizations are providing for similar studies within your industry or field. Offering compensation above the norm can make your study more attractive.

Incentive structure

Consider offering performance-based incentives in addition to a base compensation. For example, you could provide bonuses or extra rewards for participants who provide exceptionally insightful feedback, complete specific tasks or refer friends and family to future studies.

Travel compensation for in-office studies

When conducting studies in your office or lab, you should consider compensating participants for their travel to and from the event. This compensation can expand the catchment area (and, consequently, the target group) if the incentive proves attractive enough to entice individuals from outside to make the journey because it is worthwhile.

Non-monetary incentives

In addition to monetary compensation, consider non-monetary incentives that might be particularly appealing to these participants. For example, access to exclusive industry insights and memberships, networking opportunities, or early access to your product or service can be tempting.

‏‏‎ ‎

Remember, mastering test user recruitment is an ongoing journey. These tips should help you navigate the common pitfalls and ensure smoother, more productive UX studies. Happy recruiting!

About the Author
Maggy Mächler

Maggy is a Senior B2B Marketing Manager with a background in business psychology. As a member of the UX community, Maggy shares her extensive insights in various formats. She regularly organizes user research webinars, podcasts, guest posts, and events and writes inspiring blog posts. Her goal is not only to improve TestingTime's platform but also to inspire and enrich the entire UX community.

Want to receive UX Research Meetup & Event guides, new content and other helpful resources directly to your inbox? Enter your email address below!