Scaling the Impact of Research in an Agile Organisation

Ania Kaliska at UXDX EMEA. Video: https://youtu.be/H2s09Z1iDfs

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Allegro and its UX team

[00:00:00] Hello everyone. My name is Ania Kaliska, and I am Research Ops at Allegro, the biggest European marketplace. Today I would like to share with you some thoughts about research operations and its impact on organizations operating in a fast-paced agile [?] environment. But before we dive into ResearchOps, I would like to shed more light on Allegro and our story with ResearchOps.

[00:00:22] Allegro is Poland's favorite marketplace, with a 22-year history. From the beginning, Allegro has focused mainly on customer experience, including the best prices, a wide selection and incomparable shopping convenience. Each month, 22 million users shop at Allegro. There are over 130,000 sellers who offer 250 million offers, and that's quite a lot, I think. The reason I talk about these numbers is that they symbolize how we have focused on UX in our business almost since the beginning of Allegro.

[00:00:58] The support of our customers, participating in research and giving us constant feedback, led us to be the most recognizable commercial internet brand in Poland. I would say that we are like Adidas for sport shoes, or Pampers for diapers: the synonym for shopping online in Poland.

[00:01:15] Now I would like to introduce you to how the UX team looks at Allegro, and why we decided we needed research operations. As I said before, we really care about our customers' voices, and that's why Allegro started focusing on UX and research early on. The first UX research professionals joined Allegro over 12 years ago, and through those more than 12 years of the UX department being created, our team grew from five to more than 130 design and research experts. We started small, with the roles defined as UX generalists and usability testers, and as the Allegro group grew, our team also had to follow. In terms of that, I think I can say with full reliability that we now have the biggest UX research team in Poland, and we are still growing.

[00:02:11] We are at the point where data drives our business, and research became the fundamental piece of the design process. With such a big team, the number of projects is even bigger, because each quarter our research team conducts at least 100 research projects. That's literally a big number. And as you see now, the scope of our work was what showed us the only best solution: research operations.

[00:02:38] That's where I, as ResearchOps, kick in. As a big team, we needed someone who would take care of people, processes and tools, and I was given the opportunity to pick up the gauntlet and support Allegro's UX research team to become more agile, flexible, and first of all more effective in research.

What ResearchOps is and why it matters

[00:02:57] What is ResearchOps, you may ask? I think the following definition by Kate Towsey, one of the creators of the ResearchOps community and one of the biggest contributors to it, describes it best. The idea of ResearchOps focuses on the people, mechanisms and strategies that support the visibility of research across the design process. It supports the research teams in research delivery, providing effective tools and processes, scaling research impact across the organization, and also research organization and democratization.

[00:03:34] Probably most of you now ask yourselves: okay, what's the difference, what's the deal, and why should we care? Unfortunately, research is sometimes the most underestimated element of the design process, and the part that we tend not to care about as much as we should. But as we all know, UX is nothing without user research. It takes a lot of time to do the research, it costs a lot of money, it takes a lot of resources, and sometimes it can turn the design upside down. But the smaller the business is, the bigger the implications of believing that we know all about the users based on our own assumptions. That's a one-way ticket.

[00:04:15] Working in UX for a few years now showed me that it changed how companies turned to research as a fundamental piece of the design process while designing services and products. Getting back to ResearchOps, to give you a short introduction to this concept, I would like to go back to 2018, when a group of UX researchers carried out a series of workshops to systematize research's position in the design process. The community, now called the ResearchOps Community, created a map of the research activities that support design at all stages. This map, created by the team of experts, was to define when and where in the design process operations connected to research could make the design process more efficient.

The components of the ResearchOps map

[00:05:08] The key components of the map focus on budget management, which we could summarize as supporting the organization in financial resource allocation and research budget optimization. Tools, I think, are one of the most important assets in terms of research operations and optimization. ResearchOps focuses on helping find the right tools, supporting researchers in effective delivery of research findings at any time, generally. Asset management is defined as all of the tasks connected to raw data storage: its security and findability.

[00:05:46] And this one is the fun part of the ResearchOps job. Internal communication focuses on research socializing and evangelization. It also contains all of the activities connected to the reporting part, the fundamental research deliverable. This one is strictly bonded with the previous one: knowledge management makes research findings visible across the organization and supports researchers in their work in terms of reporting insights.

[00:06:16] Good guidelines and templates make research more approachable. This part makes the researcher's job repeatable, for researchers, who can focus on the content and not the structure of the research reports, and for the product team, all of whose members know what to expect from the research findings and these artifacts. It's all about making research knowledge more consumable.

[00:06:43] ResearchOps is also responsible for knowledge sharing internally and development of the research team members. This component focuses mostly on researchers' development: finding the right trainings and creating development opportunities for research team members. This all gets research in motion, and the team can be better.

[00:07:04] Bonding is also a strong part of the ResearchOps job, and this role, in my opinion, should focus primarily on people. Whether it's onboarding new members of the team, celebrating small or big successes on the team, or celebrating birthdays, that's all also the ResearchOps job. ResearchOps is also responsible for all of the company and team events that involve the research team: to organize them, find the space, and find the right time for each member to have the ability to attend. That's also, I think, for ResearchOps.

[00:07:44] The next one is governance, in my opinion one of the crucial parts of the ResearchOps job. Assuring the data is protected, and that every part of the research project is conducted in line with the law and internal regulations, is really important, as well as research ethics and privacy.

[00:08:04] Governance is strongly connected with participant recruitment. Making this process simple yet efficient is crucial for seamless research, no matter whether the ResearchOps job is to recruit participants, or just to focus on the process and conditions of the recruitment done by the researchers themselves. A detailed approach to recruitment and incentives for participants is fundamental for research success.

[00:08:33] And research spaces are all about creating a supportive environment for research. Nowadays, after COVID, most research is done online, but in fact labs or meeting rooms are still a thing, and that's also part of the ResearchOps job.

[00:08:52] As you see now, ResearchOps is a complex concept, yet it's rewarding to implement. It supports the research process end to end and lets researchers focus on what they have to do, just doing research, cutting off all of the distractions and making the processes smoother. In the end, ResearchOps supports the full design process. As you know, each of the following stages of product and service design takes the research deliverables as the base for further actions, and making it simpler and quicker to do the research optimizes the design process itself. In my opinion, the UX maturity stages [?] by Nielsen Norman Group summarize how companies should approach the design process, and why we should focus on making research more accessible and optimized.

How Allegro set up research operations

[00:09:42] Now, getting back to our Allegro approach: how do you implement ResearchOps in a well-established UX team of over 130 people? We decided to take a closer look at the current place of research in the design process at Allegro. As you know now, our business is complex and our team is enormously big, so we got to the point when we needed to adjust and synchronize the flows of designers and researchers. We knew that there were some gaps that held us back sometimes, and guess what, it took us a lot of thinking.

[00:10:21] We decided to dive deep into the ResearchOps map and create our own approach to organizing research operations. We decided to simplify it and adjust ResearchOps at Allegro to our current needs. By this we mean we specified four areas we'd like to focus on. In terms of that, I was lucky to work closely with my manager, Alina Magovska [?], who was, by the way, a UXDX speaker last year. Together with Alina, we started building research operations based on our key value: people.

[00:10:57] So it's no coincidence that we decided to focus mainly on the team, because this is the center, the heartbeat, of our organization. On top of happy researchers, we put the tools and processes that allow us to do the research effectively. After that, we focus on sharing the research deliverables. We want them to be visible and accessible across the organization, and later on I will also say a few words about the metrics. And last but not least, research evangelization. That's the key to correct understanding of the value of the research and the insights across the organization. As ResearchOps, I am also responsible for supporting research visibility and impact in the company.

People, processes and tools, and findings

[00:11:44] To be more specific, I will share some top-of-mind conclusions from our ResearchOps process and how we set it up. As I said before, people are the center of our process. In this case, we focus not only on the team composition and the role definition, but also on building a strong brand for UX researchers across Allegro. This also supports research evangelization, of course. The core of this area is to create a supportive environment for researchers, inspiring them and giving them space for self-development, because we know that what they value most is the opportunity to grow in their work.

[00:12:28] Processes and tools are the core of getting work done effectively. This area focuses not only on developing or optimizing the process in terms of the UX research team, because we know that we are part of the big picture, the product teams. Our main aim here is to synchronize the research process with the needs of other product team members, so research is involved in each phase of design. This means that we are more efficient. Of course, providing the tools that make the research process smoother is also a big point here.

[00:13:08] Research findings are a crucial part of the work of every researcher. As I like to say, I do not believe in good products being developed without proper research. But the knowledge, and its distribution across the organization, is in many cases the Achilles heel. The ResearchOps job is to make researchers' work visible across the organization, which also means supporting knowledge distribution, for example by preparing useful knowledge bases, insight repos, or any other kind of resources that help spread the word about research findings.

[00:13:45] And last but not least, it's also about measuring efficiency. Research project process KPIs help us focus on what's most important and discover the weak points we should care about most in the following weeks and months.

[00:14:03] As you may notice, we strongly believe in the fundamental role of research, and our aim is to encourage people to become a part of it. So the ResearchOps job is also to engage others in research, by giving them guidance and tools, educating them, and supporting them at any stage. We strongly believe that the ResearchOps job is also saying no when we go too far, when we want to do research but should just make an expert decision. In my opinion, that's also an expression of care about the people and resources we have.

Measuring ResearchOps and wrapping up

[00:14:41] Heading to wrap up, you may ask: is it really worth setting up research operations in the company? The main goal of Ops is appropriate resource allocation, where and when needed. So can we measure success? How can we do it? By setting up KPIs. In the case of my role, I prefer to measure my work by three components: time, cost and satisfaction. These elements help me understand whether I'm going in a good direction.

[00:15:16] As you may know, there are a lot of KPIs you can think of in terms of measuring the UX ResearchOps job: improving response rate, optimization of research budgets [?], employee retention rate in the UX research team. Anything you can think of that connects with the heartbeat of this concept.

[00:15:38] To wrap up, ResearchOps is an interdisciplinary role that supports the team in making research more accessible and fun: someone who cares about the people, their development and their work environment, and also cares about the resources and their optimization. I hope you found my talk inspiring. Thank you very much.