ResearchOps: Creating a System to Scale User Research
Need help to keep up with the increasing demand for user research? You're not alone. Many companies are seeing the value of user research and have scaled up their UX Research teams. However, when there is no structure to manage how the teams scale, things can quickly become disorganized.
In my 5 years at IBM as both a researcher and a Research Operations specialist, I saw firsthand how vital a ResearchOps practice can be. When I started at IBM, we had about 15 user researchers. In the span of six months, we numbered at 60. Within a year, we had over 100 researchers. Without a ResearchOps team in place, user research can easily get lost in the shuffle and not have the impact it should on product development. We found that to be true. Many of the practices that worked when fewer of us just didn’t scale. In addition, we found it harder to communicate and share knowledge.
At this point, focusing on Research Operations (or ResearchOps) has helped us tremendously. And it can be already helpful for smaller teams as well. ResearchOps helps scale the research craft across a business and takes away time-consuming tasks from the researcher.
In this article, I’ll give an overview of ResearchOps and how it relates to user research, along with some tips to get started with ResearchOps.
- What is Research Ops?
- The 8 Pillars that Connect User Research with Operations
- The Importance of ResearchOps
- Knowing whether your organization is ready for ResearchOps
- Role of a ResearchOps function
- How big should a ResearchOps function be?
- Getting Started with ResearchOps
- Prioritize what areas to address first
- Establish measurements of success
- Constantly re-evaluate
- Final thoughts
What is Research Ops?
Similar to the establishment of DevOps, DevSecOps, and eventually DesignOps, Research Operations arose from the need to provide operational support to research teams. For this article, I’ll use the corresponding shortened notation ResearchOps. Let's look at the definition of ResearchOps crafted by the ResearchOps Community: "ResearchOps is the mechanisms and strategies that set user research in motion. It provides the roles, tools and processes needed to support researchers in delivering and scaling the impact of the craft across an organization."
When a research team of 5 grows to 25 in three months, the communication between team members rapidly changes. Now imagine 5 researchers growing to a team of almost 100 within a year. As I mentioned earlier, that’s what happened to us at IBM: When we were on a research team of 5, it was much easier to communicate with the rest of the team. We could have regular calls and discuss things in detail. Now when there are 100 of us, it becomes a lot harder. Our organization has grown so that we have different managers and work on very diverse projects.
Imagine trying to establish consistent processes and frameworks across this large team on top of your regular job of doing UX research with all of its demands. It's almost impossible to do the operational and administrative work required while keeping up with your work.
I started doing a lot of ResearchOps-type work in my role as a researcher. Eventually, I joined our ResearchOps team to help scale up the practice and remove friction from our UX researchers.
The 8 Pillars that Connect User Research with Operations
In the early days of the ResearchOps community, there was a big movement to define what ResearchOps is (#WhatIsResearchOps). A ResearchOps Slack channel was created, and many conversations took place around this topic. However, to formalize this work, a group of volunteers created a survey and ran workshops worldwide to work on this definition.
I participated in the San Francisco workshop and shared the struggles I was going through. I remember talking about how frustrating it was to advocate for the value of research rather than having my research last beyond one particular research project.
The culmination of this work resulted in the eight pillars of user research. The pillars involve what user researchers or "people who do research" (PWDR) care about.
1. Environment
The first pillar, environment, involves the overall landscape in which user research should happen inside the company. It refers to the people and silos that make it challenging to find the right advocates for research.
Environment includes education and helping others understand the value of research. It encompasses getting buy-in and experiencing pushback on research findings.
I’ve seen our environment change considerably over the years. The more value the user research team shows to the business, the more buy-in we get. Finding advocates helped grow investment in our operations.
2. Scope
Scope includes how and when research should happen as well as what research methods should be used. It involves prioritizing research, determining how often to share insights, developing protocols, and selecting research methods. It also involves creating a cadence when to hit different milestones, such as how often to set research planning meetings and sharing of insights.
In my experience, the scope depends entirely on the type of project. Certain teams work at different paces, so it’s important to know your audience and fit within their cadence.
3. Recruitment and admin
Participant recruitment and the administrative work around that can be one of the biggest challenges for user researchers, and one of the biggest areas that need ResearchOps support. It involves all the logistics of participant recruitment, from recruiting to scheduling, panel management, incentives, and paperwork.
I could write articles and articles about this topic, since it was my main focus as a ResearchOps specialist. At a big company, there are a lot of internal and external hoops to jump through when it comes to recruiting. There are legal considerations such as consent forms along with internal relationship building to get access to customers.
4. Data and knowledge management
As teams grow and change, many times valuable research insights get lost, leading to studies getting repeated. It's important to figure out how to ensure that the same studies aren't repeated. It's also essential to make sure that research gets consumed by the right audience.
There's almost nothing worse to a researcher than their data not getting used. I remember having new teammates come in and ask me to run research that I’d already run before they joined. I had to do a lot of work to convince them that we’d already found out the answers to their questions.
5. People
When it comes to people, it's important to remember that not just UX researchers do research. There's often such a need for research that it might be done by designers and product managers.
As a research practice grows, it's important to create a mature community of practice to support the staff. That means developing career paths and having leaders within the research community. I personally love that my current company has a dedicated individual contributor path for researchers.
6. Organizational context
It's important to consider the overall organizational context, including various internal and external constraints. These constraints might be: space, time, resources, budget, return on investment, business constraints, market forces, and organizational maturity.
These constraints are things that most researchers hate dealing with. Sometimes it falls on managers to do this. However, when it comes to determining an overall strategy for an organization, it’s helpful to have an operations team managing things like the budget because then there’s not duplicate spending for similar things. For instance, two different managers requesting recruitment funds and finding out that it would have been easier to pool resources.
7. Governance
Legal and ethical considerations are at the heart of the governance pillar. It involves assessing risk, privacy laws such as GDPR, legal guidelines, information security, consent, and ethical considerations.
Without ResearchOps, different teams manage governance in different ways. Some do a better job and adhere to regulations. Others do not and create risk for the company. Our ReOps team has created standard processes and documentation to help mitigate these risk factors.
8. Tools and infrastructure
This final pillar involves looking at the systems and tools needed to complete projects. It consists of evaluating tools and procuring them. It might involve figuring out hardware and software needs. Maybe your research team needs a physical lab or a research repository. All of that falls under the tools and infrastructure pillar.
Before we had a ReOps team, and we wanted to get a specific tool for our org, a researcher might have had to take up the mantle of advocating for a new tool. That included getting the budget for it and going through the company’s approval process. Our researchers are so glad that the ReOps team handles this type of work now.
We do regular inventories of our team's tools and research what’s new in the industry to ensure we’re providing our team with the best possible toolkit.
The Importance of ResearchOps
The eight pillars of user research encompass the daily work of a researcher. Some of these tasks must be done by a researcher. However, some of these tasks are better suited to a research operations function. Therefore, a ResearchOps function becomes that layer on top of the user research pillars to bring everything together. It creates a unified approach that breaks down barriers within companies to make user research work more streamlined.
If an organization does not have a ResearchOps function, it forces other roles to take over the ResearchOps responsibilities.
When these other roles are forced to do ResearchOps, it may take longer because they don't do them often. For instance, it might take a product manager a long time to create a screener to recruit participants, select participants, manage participant incentives, and manage consent forms.
Not only will they not do the job as efficiently as someone who does ResearchOps, it also costs more because it's time the product manager isn't spending doing their primary job.
Moreover, when trying to do research at scale, if a team wishes to be efficient, it's vital to have processes in place. You want your researchers to have time to do research!
Knowing whether your organization is ready for ResearchOps
Personally, I think that if your team has more than 5 researchers and you anticipate growing more, you might want to start thinking about introducing a ResearchOps function.
If you don't have the budget to hire a dedicated ResearchOps manager, consider seeing if someone on the team is interested in taking on ResearchOps as part of their existing role.
Role of a ResearchOps function
A ResearchOps function ensures researchers have time to do what they do best: research.
It all goes back to the 8 Pillars of User Research. ResearchOps removes the administrative and operational tasks within each pillar to enable researchers to focus on their job.
These tasks might include:
recruiting participants
managing incentives
identifying data collection strategies
onboarding new tools
managing the budget
ensuring data privacy policy compliance
managing consent forms
So summarized:
„ResearchOps helps scale the research craft across a business and takes away time-consuming tasks from the researcher.“
How big should a ResearchOps function be?
I've seen ResearchOps manager job descriptions that expect the ReOps manager to do almost everything related to operations within the 8 Pillars of User Research. It seems like A LOT of work, but it also depends mostly on the size of the company and research team. Other factors like the number of PWDR, the general UX maturity, and the company’s approach to research democratization also play a role, but nonetheless from my experience the strongest factor is the number and size of the research team and organization.
It might work if a company has 7-15 researchers and hires one ResearchOps manager. However, if it's a team of 50 or more researchers and you want to hire just one ResearchOps person, it might take longer to see the results you want.
The size of your ResearchOps function should be somewhat proportional to the size of the team they're expected to serve. In my opinion, you should strive to have one ReOps person for every 7th researcher. That's probably only possible for some, but it would be something to aim for.
Getting Started with ResearchOps
There's no one definitive area everyone needs to tackle to get started with ResearchOps. Every organization is different. Departments run differently. You have different stakeholders. However, from what I’ve experienced, there are a few best practices for getting started:
Prioritize what areas to address first
As a ReOps function, you should be researching your own stakeholders. Start by researching your own team. Maybe conduct stakeholder interviews or a survey. Or both!
You want to identify quick wins and your organization's most significant pain points. From there, prioritize accordingly.
You might consider using the prioritization grid technique to sort out the value to the user vs. the feasibility to your team.
Develop a 30-60-90 day plan of what you can accomplish within a short amount of time and plan for projects that will take longer, but have a high impact.
For instance, our ResearchOps team discovered that recruitment was one of our biggest pain points and that researchers didn’t have a clear idea of the processes within our organization. As a quick win, we created a simple playbook that outlined all of our processes that we had in place so far. As a larger project, we created a dedicated role for recruiting.
Establish measurements of success
Over the course of your work, you'll want to benchmark your ResearchOps progress over time. For example, every quarter, you might want to send out the same survey to the researchers you serve to see how different areas within your practice have improved. It'll help you not only advocate for the value of the work that you're doing, but it'll also help you make sure that you're meeting the needs of your team.
For instance, we discovered improvements in our researchers’ satisfaction with our external panel recruiting options, but learned we still had ways to go for building up our internal panel options.
Constantly re-evaluate
Pay attention to what’s working and what’s not working. Through your benchmarking, you'll be able to quantify this more easily. You'll have actual data around this.
Decide if you need to continue what’s working or if there’s a more pressing issue. For instance, let's say your big project is participant recruitment. You've planned a lot of your work around this. You still have issues to tackle.
If you discover that the team's needs...
fit within the work you have planned, then you're definitely on the right track!
are a little different than what you have planned, you might want to re-prioritize what you address first within that area.
are in a completely different area, that might be a bit frustrating, but it's also essential to know. For instance, you could discover a huge data privacy and compliance issue that might take precedence over what you're doing.
It's important to be flexible and willing to meet the needs of your team, and this means to re-evaluate and change directions from time based on the team’s needs
Final thoughts
ResearchOps provides the structure needed to support researchers in delivering and scaling the impact of user research across an organization. ResearchOps enables the eight pillars of user research to be successful because it helps break down barriers inside an organization. The size of a company’s ResearchOps function should be proportional to the research team they’re meant to serve. To make a ReOps function successful, it’s essential to work with your research team, be flexible, and constantly re-evaluate. All in all, ResearchOps helps keep everything organized so that users remain the focus of all product decisions.
Rachel is a lead user experience researcher at IBM and leads competitive evaluations across various products within IBM Software. Previous roles include ResearchOps, product management, and UX design.
The above article is personal and does not necessarily represent IBM’s positions, strategies or opinions.
