How To Democratise The Power Of Data & Insights

Ananda Nadya at UXDX Community: SE Asia. Video: https://youtu.be/8ef0QaPYseU

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Introduction and Tokopedia

[00:00:00] Thank you for staying here. I would like to introduce myself first. My name is Nanda, and today I would like to share with you how to democratize the impact of data and insights through UX research. This is a brief introduction about myself. Other than telling you my name, I will also tell you where I work: currently I'm working at a company called Tokopedia. If you have any questions, or you are interested in talking with me after this event has ended, you can approach me through my personal social media or my email, which are listed below.

[00:00:49] After introducing myself, I would like to also tell you about where I work. As I said before, I work at a company called Tokopedia. It is actually one of the biggest e-commerce companies in Indonesia. It was established in 2009 by William Tanuwijaya and his best friend Leontinus Alpha Edison. Tokopedia has now undergone a merger with Gojek, so as you can see in the picture, the name is not solely Tokopedia anymore; it has become GoTo, or Gojek Tokopedia. As of 2021 the company already had more than 4,000 members, which includes probably hundreds of product designers and product managers. Since it is an e-commerce platform, the platform itself has accommodated about 11 million sellers, so that they can reach their buyers more easily, and currently the platform hosts about 100 million users.

[00:02:07] If you are curious about what Tokopedia looks like, this is a glimpse of the platform. As you can see, it's just a typical e-commerce platform, but it is marketed towards Southeast Asia, especially Indonesia only. You can see in the pictures here a glimpse of the app itself and how it functions.

[00:02:37] Okay, now I'm going to go through what I'm going to give you. Today we will discuss several things. The first one is about making insights meaningful. Second, we will go through the UX research maturity stages, and of course we will also talk about the process of going through the maturity stages. And last but not least, I will also give a summary of the presentation and what it means for you who have stayed and listened to this presentation.

Making insights meaningful

[00:03:23] Now I will start with the first topic, making insights meaningful. As you can see, my team in Tokopedia is RAD, or Research Analysis and Development. It was established in 2017, and since the time of establishment, we in the research team have always aimed to create meaningful insights, so that we can help stakeholders reach their goals. We want them to be able to derive actionable insights from the research results that we present to them.

[00:04:06] Now I'm going to tell you a little bit about how much our team has grown over the course of five years. As you can see, the team, the team members I mean, have grown from a team of five in 2017 into a team of almost 30. We also had a shift in direction. We started off as a team that is request-driven, which means that stakeholders or people within the company usually request things from us. But as time went by, we shifted our direction to be more initiative-driven, which means that we try to give the initiatives back to the stakeholders and pitch our ideas, so that we can help them create direction for the company. I'm going to delve a little deeper into how we made this transition over the course of five years.

The UX research maturity stages

[00:05:16] Before I talk step by step about the changes, I would also like to show you the research maturity stages, because the research maturity stages really affect how the team's strategy shifts from one stage to another. There is this research maturity model from Afore [?], established in 2016. Basically, this man divided the maturity stages into four. The first is laggard, the very early stage, in which the research team still focuses on user feedback, but the company doesn't really trust the research team yet. The second one is the early stage. This stage consists more of ad hoc projects, and usually research is used for preventing mistakes and evaluating things.

[00:06:23] The next one is the progressing stage, which already includes more exploration. And last but not least is the mature stage, in which the company already trusts the research team and gives them the freedom to explore many things and to forecast trends that may help the company shape its own strategies.

[00:06:52] Now I'm going to tell you about the process of going up the maturity ladder, which consists of those four stages. When we talk about progression, I will take the example of what happened in my company, to give you a better depiction of how we went. In 2017 we started as a team of five, and we were still in the laggard phase. One year after, when we grew to ten researchers, we shifted into the early stage. After 2018 we shifted into the progressing stage, in which we already had around 28 researchers, and we also had product design coordinators to help us with our operational research needs. And then from 2022 onwards, our team has shifted to the mature stage of a UX research team, in which we are moving to a hybrid team type, which combines embedded and centralized when it comes to structure.

Laggard: gaining trust and awareness

[00:08:05] This is a preview of what we went through in 2017, when we were still in the laggard phase. As you can see in the picture, it's a centralized team structure, because we were still low on bandwidth and resources. Therefore our focus in this stage was to gain trust and awareness from the stakeholders. Because the research team in Tokopedia is placed under the product design team, we received a lot of requests from designers from different product lines, for usability testing and other evaluative things, and the researchers would help them conduct the research.

[00:09:03] This is a very nostalgic photo of how we started. We gained trust and awareness by basically doing 90% directional projects, which means people requesting from us what to do, and it was mostly focused on design-related projects. In terms of methodology, we conducted more usability testing, because we wanted to evaluate the designs that the designers had made, and we also conducted several heat maps and interviews to help give better context on what the design was all about. The prominent stakeholder in this stage is the product designer, because we dealt with design a lot during this time. And we also tried to spread awareness of our research findings and research work through several methods, such as a newsletter and a pain point database, and we also invited stakeholders to the sessions.

Early: company-wide awareness

[00:10:14] Now I'm going to shift into the next stage, which is the early stage. One year after we were in the laggard phase, we evolved into the early stage. This time we already had around 10 people in the research team, and the focus was to create company-wide awareness, especially at higher levels. Previously we gained the trust of one stakeholder, the designer, and in this stage we wanted to gain more trust from other stakeholders as well. In terms of structure, we implemented a semi-embedded team structure, in which a researcher works together with the stakeholders to collaborate better.

[00:11:11] Here is how we work. Stakeholders of different types, and it can be more than just product designers, for example product managers, or people from the CX team, or data analysts, can request a project from us. Within our team, we have a person who acts as a captain, whose responsibility is to handle the backlog. After receiving the request, the captain is responsible for assessing the importance and also the availability of the researchers, and then, if possible, the captain will assign the task to the available researchers who are most suitable to do the work.

[00:12:06] This is a glimpse of our team members in the early stage. As you see, there was a shift in the type of projects that we do. We now conducted more foundational research, but we didn't abandon directional research. When we talk about directional and foundational, directional usually deals with something that is already concrete, such as design evaluation or concept evaluation, while foundational is more like abstraction, which deals with exploratory ways of uncovering things. Because we wanted to gain company awareness, we shifted our focus and type of projects towards foundational ones as well, since we wanted to give them insight on what they can go after, instead of only evaluating what is available.

[00:13:12] Now we also used more varied methods. We didn't only dwell on usability testing; we also did interviews and surveys, diary studies, card sorting and focus groups. As you see, the stakeholders also really varied, as I mentioned before, and the way of spreading awareness also evolved. Not only did we send a newsletter and create a pain point database like before, now we actively invited stakeholders to the research sessions, so that they could contribute to the data collection and the discussion. We also held several research events. For example, we had an offline exhibition which showcased the findings of our team, because we wanted them to actually be aware of what the research team is doing.

[00:14:11] We also had a research internship program, which is exclusively for people to enroll in the research team, and we educate these interns to be researchers in their respective areas, because we aim to source local students who are probably interested in UX research but don't live in Jabodetabek. And we also created a research repository, so that we can store our previous research findings and people can easily find them.

[00:14:51] These are some of the things that we did in the early stage. We also contributed by doing research at company events. For example, at Makerfest, an event in which the company helps local sellers promote their products, we did guerrilla research, so that we could get insights from multiple locations, especially those around the central cities, and get insights from rural areas as well. The second one is the internship that I told you about. The third one is when we do guerrilla interviews, which are more spontaneous. And the fourth one is a glimpse of how we work together with stakeholders, with them participating in the research process.

Progressing: strategic research and impact

[00:15:52] Now I will shift into the progressing stage. In 2019 the team grew into a team of 20-ish people. Because the team size was already quite big, we focused more on conducting strategic research and on the impact that we made through our research. As you can see, previously we had a semi-embedded structure, which means that we may be embedded in the tribe, but sometimes we still work within the research team. Now we were shifting to an embedded team, which means that we are required to attend several meetings within the product line that we are assigned to, for example design sprints, product town halls, etc., so that the researcher and the stakeholders can have a more comprehensive understanding of the product, which leads to effective collaboration.

[00:16:57] There was also a shift in our flow. We started from the prioritization roadmap made by the project team, and that document is used by the stakeholders to request projects, which are divided into ad hoc and consulted projects. An ad hoc project is a project dealing with a certain research topic that the researcher is assigned to. For example, if I were in the logistics product area, I would handle every research project regarding logistics. However, we also accept consulted projects, in which a researcher doesn't have to engage through the end-to-end process. This kind of project is really fast, and the researcher is only responsible for assessing the research materials that the stakeholders will use, because in a consulted project, stakeholders are encouraged to conduct the research by themselves, in order to gain a better understanding of their research and better knowledge of their product.

[00:18:25] This is a glimpse of our team in 2019. As you can see, we mostly focused on foundational projects, because we aimed to shift into more strategic research. And of course, when we shift into something more strategic, the methodologies that we use also grow. We conducted various methodologies, such as ethnography. We also have a unique methodology called speed dating, or speed chat, and we conduct co-creation as well, so that we can work better with participants in creating a design. The stakeholders also varied: now they include media intelligence, business development, etc.

[00:19:22] In terms of gaining influence, we tried joining team rituals, as I mentioned before, like participating in the project town hall or design sprint or co-creation. We also created several creative activities, such as research podcasts, and we held insights exhibitions with interactive games, so that people could be more aware of the research. The unique thing here is that we also created an overview reflection report, a new annual research report sent by the research team to the whole company, so that they will be aware of what kind of research we have done that year.

[00:20:14] During this time we aimed for better research democratization, and because we started accepting consulted projects, I'm going to give you an insight into who our stakeholders are and what we encourage them to do in terms of research. For example, we work with product designers, product managers and business development a lot. Therefore we encourage them to conduct simple research that may be useful for their jobs, such as competitive analysis, usability testing and surveys, and also interviews. We do this by involving them in our data collection, and we are also involved in their data collection, so that we can give feedback on how they should conduct it, what their strengths and weaknesses are and how they can improve. We also hold internal training, in order to give them insight into what research is supposed to be and how it is supposed to be conducted.

[00:21:25] Within this year, this was our research process before, and after it changed, it became like this. Now we don't only take starting points from requests or initiatives. We also start from the existing documents or vision documents that the team has made, and from internal data. We work together with data analysts to gain insight into what is trending, and we combine it with the planning that the respective team has made for the fiscal [?] year. If it is something that could be done to gain more opportunity within the company, we will try to create an initiative out of it. Not only that, we also try to actively measure the insights by holding alignment sessions after the research is done, so that we can measure the success of our research.

[00:22:36] This is the kind of cross-team collaboration that we hold within the company when it comes to research. Stakeholders are expected to be involved in the pre-development, development and post-development stages. Through this collaboration, we hope that our stakeholders will realize the importance of research, and the importance of using data before making decisions.

Mature: influencing product and company direction

[00:23:11] Now we are talking about the last stage, which is the mature one. My team is already in this stage, and hopefully we'll stay in this stage. Since 2020 our team has been considered a mature team, because we have more than 20 researchers. In 2020 we were almost a team of 30. Therefore our focus shifted to influencing product and company direction, and to the growth of the team itself, because we still aim to hire many researchers in the future, and we would like to establish a system that will enable the team to work more effectively, so it can be more impactful for the company.

[00:24:10] There's only this change, in which research leaders develop prioritized projects to be done in the next quarter. Other than the ad hoc projects and the consulted projects, now we also have these prioritized projects. They are usually plotted by the leaders and assigned to a certain researcher with a certain expertise to conduct.

[00:24:42] This is how we influence product and company direction alongside team growth. As you can see, in this stage we only conduct 10% directional research. The rest is foundational research, which consists of UX research, product research and market research. We use numerous methodologies now, and because the pandemic hit and has been very impactful on how we work, we mostly conduct everything online. As for the stakeholders, we not only get requests from project managers, designers and business people; we also get requests directly from C-level and VP level, and from people in strategic roles.

[00:25:42] Because our audience is now becoming more strategic, we hold several rituals, such as a monthly insight presentation to the chiefs, and foundational and innovative research, in which we explore the possibilities of several products that are not yet here in Indonesia and assess people's acceptance of the concept. We also transformed our OKRs, and we triangulate the data with the data analyst team, so that our insights can become more comprehensive. We are also still producing the end-of-year report highlighting impact, so that people will not forget what kind of impact the research team has made.

Tips for each team size

[00:26:36] Now we are at the last presentation topic, which is the summary of what I have just told you. Here are some tips if you want to shift your team from directional to more foundational research, so that your team can be more strategic in the future. If you are a team of probably five people or fewer, you may want to focus on building awareness first, because everything that you do probably won't matter if you don't get acceptance from the people that you are working for. In this stage you may want to focus on befriending people and showing them the work that you could do to help them. Maybe you can try to assist them in their day-to-day job, and you are expected to communicate with them closely, so that you can assess what they need and actually help them overcome their problems.

[00:27:54] If you are a team of six to ten people, you may focus on building awareness at a company level and with higher-ups, because now you have more bandwidth than before, and you probably want to challenge yourself to do more complex projects. This should be done in order for you and your team to increase your exposure and the strategic impact that you want to show other teams, so that you will have a lot more freedom to act as a research team. You can also create company-wide events that might help you gain exposure. You may also want to start documenting your insights and reports, so that people can find them if they want to search for a specific research project, and so that you don't lose the things you have already done, in order to avoid redundancy in the future.

[00:29:05] Next, if you are a team of 11 to 20 people, you may want to focus more on strategic research and impact, because now the team has grown significantly in size. Now you may want to prioritize the research that you do, because in this stage people already trust you to do your job, so there must be many requests coming in. Therefore you have to find a way for people to be able to do the projects while also maintaining the strategic way of thinking and way of doing on your side. This is where you can start to democratize research, by encouraging the stakeholders to do their own research. You may also want to regularly share insights in interesting and creative ways, such as building podcasts, or sending yearly reports, or holding quarterly meetings, things like that. And if possible, you may want to hire people who will handle operations, because it will be hard for you to focus on so many things at once, so you had better find a dedicated person who will help you with the operational things.

[00:30:33] Now the last one: if you're a team of more than 20 people, you may want to focus on influencing at a bigger scale and on growing the team itself. In this stage you may be expected not only to conduct research on what is already presented, but also to find opportunities that people haven't yet discovered, in numerous ways, because in this stage the research team itself has become the key to forecasting trends for the future. Therefore in this case you have to triangulate the data better and find creative ways of running your research. You may also want a more systematic mechanism in order to strengthen your team, since you want to be able to maintain the quality of the research that you do, and you may want to grow your team to become bigger and more strategic in the future.

[00:31:51] So yeah, I think that's pretty much it. Thank you very much for listening, if you have any questions.