Empowering Researchers in Product Teams

Om Tandon, Duaa Gettani, Vaseem Khan, Alexandra Lung at UXDX EMEA. Video: https://www.youtube.com/watch?v=aEJmuRZ8_1M

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Introductions

[00:00:03] Vaseem: Hi, hi everybody. My name is Vaseem Khan. As said, I'm a data leader from a company called Fullstory. Really excited to be here. I think, firstly, just how good is it to be in front of people? Pandemic gone, it's just good to not live in pajamas, be on Zoom. So big hurrah to that. Really excited to talk about empowering researchers in product teams today, but before we get started it'll be great for our panel to give quick introductions in terms of their role, their focus areas, and then we'll build from there. If I can start with you.

[00:00:35] Om: Thanks, Vaseem. Hi everyone, great to be here. This is my first in-person conference after the pandemic, so awesome to see so many faces. I've been 15 years in gaming, guys. Basically worked across platforms, starting from PC, console, casino, Nintendo Wii, and the last eight years in the mobile space. Currently I'm heading market intelligence, UX and user research at Wildlife, which is one of the top 50 publishers of mobile games in the world. And in my journey I've been very fortunate to work with a lot of IPs, so some of the games I've worked for include franchises like Star Trek, Ice Age, My Little Pony, Disney and so on.

[00:01:22] Vaseem: Thank you, thank you.

[00:01:23] Alexandra: Hello everyone, I'm Alexandra Lung. I'm CPO at Uptime, which is an IoT French startup, and just before that I was CPO at a signature group[?] — basically there were three companies that merged together to be a European leader. And before that I was leading all the product and design teams at Aircall, which is one of the latest French unicorns as well. And before that I had another 10 years in product and design in different companies.

[00:01:51] Duaa: Hi everyone, my name is Duaa. I'm currently a senior researcher at Square, previously coming from academia. I think we've spoken a little bit about that academia to UX research pipeline. I'm currently working within Square Banking, which is a program where we provide banking services for small businesses, specifically those small businesses who currently process their payments through Square.

How the teams are structured

[00:02:19] Vaseem: Amazing, thank you. So today's topic is, how do we empower researchers in product teams? It's a topic that's close to my heart. I've worked with both researchers and product teams for many years, and I've lived and breathed both the cultural as well as the data transformation that research teams and product teams have taken. What's interesting for me is to see the transformation in terms of the team structure: how teams have adapted as companies have grown, and how business priorities have shifted. So I think a good place to start, because we've got such a diverse panel from different companies, it'll be good to understand how their teams are structured, and how their teams have shifted as teams have grown. So maybe if I start with you: it'll be good to get an understanding of how your teams developed, how your team is structured, and then we can build from there.

[00:03:11] Om: Sure, thanks Vaseem. So for me, guys, it's a bit interesting because I have almost three different teams. One of them is a centralized team which is made up of user researchers, so it's more like a service team. It's a centralized team and it serves different products. So we have new games, which are products under development, and then there are these live game operations, which are products which are mature and out in the market. So this centralized team serves the needs of both the business divisions. But then I have a UX team which was also earlier a centralized team, but then we had a reorg and now they're embedded into each product.

[00:03:49] Om: So it's interesting, because for me what's important is I want to make sure that they're communicating with each other. But there are challenges to both. In a centralized team there's a lot of context switching, but in a decentralized team there's this risk of siloing teams. So at the moment I'm handling both of them.

[00:04:10] Vaseem: Just so I understand the terminology correctly, when you mean centralized, it's a shared resource?

[00:04:14] Om: Yes, across the entire org.

[00:04:17] Vaseem: Yeah, that's really interesting, and I'll certainly come back to that, because it'll be interesting to understand how you align the business priorities when you've got different teams and different products and challenges.

Decentralized teams, and who can do everything

[00:04:30] Vaseem: Alexandra, I know you've got quite a varied background. You've worked for a scale-up and you've also worked for a unicorn out in France. It'll be interesting to understand your take on the types of teams that you work with.

[00:04:42] Alexandra: Yeah, of course. So in my case I worked mostly with decentralized teams, basically having a role like a product designer that is able to do everything from the research to testing and to implementation, and they will be embedded in a team that has a certain mission, and they work daily with the PMs and engineers. What I've seen working pretty nicely in this is that the teams are very autonomous in their learning, and they can go pretty fast, and they can get closer and more easily to continuous discovery, as they have this resource that is able to do that.

[00:05:22] Alexandra: It's not always easy to have people that have these skills, especially when the companies grow, especially at a certain level, one that can be really good at the research part but also at the rest. So that's one of the challenges that come in, and also, as I was saying, with the silos and communication that we need to make sure is more fluid, and everyone has the information needed. But in my case, in the previous three companies, it was mostly product designers that were growing in time and in the teams.

[00:05:54] Vaseem: That's really interesting, and we'll certainly touch on that concept of decentralized and, as teams grow, how do you keep aligning priorities and making sure — because to some extent, in some contexts, it then feeds back into that centralized overarching theme, right? So we'll certainly come back into that.

Decentralized, hybrid, and staying connected at Square

[00:06:16] Vaseem: Duaa, I know you're working with Square Banking. Really interesting growth that we've seen with Square Banking, so I'd love to understand from your world how your teams are structured and what you're seeing.

[00:06:27] Duaa: Yeah, so we're a decentralized team. I know that we sort of have this description of centralized versus decentralized, but to some degree I feel like there's still some hybridness associated with that. So we are a decentralized team. Each of our designers and researchers are embedded within specific product teams, but obviously the ratio isn't always perfect, so you're supporting more than one product team. So in that sense you're almost in that sort of service model, centralized team.

[00:07:06] Duaa: And then we also have sort of our centralized working group, where we are all associating with each other, having meetings together, so that we're not losing that connection. I think that that's part of the challenge with decentralized teams, where you're losing that connection with other product designers and other researchers. And then with our team it's not just designers, it's creative as well, so you have content designers and our creative team as well within our design work.

[00:07:32] Vaseem: Interesting. So there's two parts to that. You kind of mentioned that the teams are decentralized and, to an extent, they're kind of working in their own silos. But obviously you are working for Square, and Square have got their own business priorities that they're trying to push out to market. So how do you make sure teams are aligned to those business priorities, whilst making sure the teams themselves are empowered in doing what they need to do to be successful?

[00:07:58] Duaa: I think for the most part, because we are embedded within each of those product teams, most of the product designers and researchers are sitting in those meetings and have that context. So that's obviously the benefit of being in a decentralized team, where you're not necessarily having to do that context switching, which is kind of the challenge with a centralized model, where you're doing that context switching quite a bit and not necessarily having the knowledge and the context that you need to support those projects.

[00:08:32] Vaseem: What do you lose with that model, then?

[00:08:34] Duaa: What do you mean?

[00:08:35] Vaseem: So in terms of that decentralization of the team from the overarching structure, I'm assuming it's quite easy to be siloed and live in that silo, as opposed to understanding the overall overarching objective.

[00:08:50] Duaa: Yeah, so our team meetings, where we are in constant communication with each other, making sure we're not repeating research. Especially with researchers, as a researcher there's that challenge of having sort of similar research priorities, because it is all under one company, and so you're making sure you're not doing repeat research and things like that. So making sure, within those decentralized models, to continue those bridges between other researchers and other designers.

Keeping everyone aligned as the company scales

[00:09:24] Vaseem: Interesting. And Alex, I'd love to hear your take, because Aircall, I think everybody knows their story, French unicorn, and I think you joined and helped scale Aircall from a very small team to a very diverse and very large team. So I'd love to understand your insight in terms of how you kept the overarching business goals present, and what limitations you saw as the team scaled.

[00:09:47] Alexandra: Yeah, I think in terms of goals, indeed it's really important, the big picture as well, just having the north star metric that aligns everyone, having OKRs, and everyone knowing what they're feeding into and what their impact is supposed to be. Then, in terms of what Duaa was saying on how you synchronize everyone, because as the company gets a bit bigger, the sync and the silos become pretty critical. We also had team meetings, as you're saying, but also just sharing that information widely with other departments, with the company as well.

[00:10:25] Alexandra: Also small things, like if I'm doing research and I learn something about someone else's product, just tagging that person or tagging that information as well, so everyone is empowered on their impact but also not being like, oh, this is not something I care about, and just throwing it away, but really relaying that information to the persons that will really take care of that as well.

[00:10:46] Vaseem: Absolutely. So what you're saying is you need constant validation and constant contact with the teams, so you are sharing ideas and making sure you're not kind of working in silos.

[00:11:00] Alexandra: Exactly, yeah. And something else that we were doing — I think it was somehow easier, I've seen that happening pretty well in small startups — is that, for example, if you only have two designers, sometimes you can have them switch roles or teams as well, and in that case both of them, in time, and then when someone else joins as well, have all the context. When the company is still pretty small that works pretty nicely, and then if someone leaves, the other person still has a view on everything. It depends how at ease the teams are with change, and sometimes, even when having product designers, they're not all super experienced, so can you actually give them the responsibility of a different team or not? But in some cases it works really nicely.

Maturity, institution and OKRs

[00:11:42] Vaseem: Interesting. And we'll certainly come to that concept of scaling, and as teams grow, how do you embrace that change. But Om, I'd love to hear your perspective.

[00:11:53] Om: Yeah, I think a lot of good information the ladies mentioned. In my case it all starts with what's the maturity of the org I'm joining. So I classify: no maturity, low maturity — when talking about UXR — and high maturity. In my experience, 70% of the companies are either no UXR maturity or low. What I mean by no is there's no existence of UXR in that company, or there is some form of UXR they do but it's not perfect. So the first thing is to gauge that.

[00:12:21] Om: In my case, how I try to structure it, just taking the example of Wildlife: it was a low UX maturity company, they had UX designers but they were not following all the processes. So for me, how the hierarchy looks is: team over people, discipline over team, and then institution over discipline. That's the way I stack it. And at the institutional level, something like what Alex said: even if I'm not there tomorrow, or none of the members are there, what is it that will guide the people? So we created a UXR human-centered design institution, and it's a combination, a discipline responsible for craft, aligning the goals of the company.

[00:12:59] Om: So we looked at the OKRs — our company OKRs, as we developed our departmental OKRs — we looked at the company vision, then we said how can we tailor our vision which incorporates that, and also the values. For example, one of the values Wildlife has is "we are fast", because they came from a startup phase almost four years back. So we were like, how do we incorporate that into the UX/UI discipline? And we're like, yeah, if our product managers need to make decisions fast, then we need to, especially for new products, find these processes and pipelines, without cutting corners, where we can help them move fast with research and design. So I think keeping those OKRs which feed into the company vision is very important, for these research and design teams.

Change management and proving value

[00:13:38] Vaseem: That's really interesting, and I think part of what you said there is the change management piece, right, in terms of going from a smaller company to a larger company. How do you manage that change? What's the best way to — because, to abuse the phrase, you're going to have technical debt in terms of people who are working the way they've been working, and it's been working, so we're going to work the same way. But obviously as the company's grown, things have changed, so that same way of working perhaps isn't the best way to carry out research. So how do you manage that process? Because obviously you're not going to manage these people out of the business, but how do you bring them on board in terms of the new journey, and what does that process look like?

[00:14:14] Om: Yeah, you're absolutely right. So change management is built into the job, because it doesn't matter — in my experience at least — small company, big companies, there's always reorgs. It might come at six months, one year, and they're like these huge earthquakes. They can move departments around, teams around, your own job role can change. I'm sure all of us have experienced it.

[00:14:34] Om: So I think the first thing is for alignment, to build traction. As you said, they're used to working a certain way, and then you're like, hey, by the way, this is our entire UXR pipeline, we should have a discovery phase, we should have our ideation phase, and we should do voice of customer or focus groups and all that good stuff. But it always comes down to proving value. So for me, I also try to create a safe place to fail. For me it's more important to get momentum rather than getting it right, especially if you're pioneering new techniques.

[00:15:03] Om: So I tell my teams — we do a lot of design thinking — we are taking these de-risked projects. For example, if I go to a team and say, hey, you're releasing this feature next month, do you want us to take it through the UXR pipeline? They're like, no, there's too much risk, I don't have the time, it's already planned for. I look at a feature which is three months, six months down the line and I say, hey, you guys have time, can we run this through our entire process and show you the value? So these are some of the ways. If you're able to do that, demonstrate value, we automatically get traction with the business.

[00:15:31] Vaseem: Interesting. So it's constantly validating, constantly showing how you can show value to bring people on board and push that journey.

[00:15:39] Om: Yeah. The only thing to add there, Vaseem, is I think it's important to see this tactical day-to-day stuff which you have to do within the constraints you have, but then never lose sight of strategy. And strategy is all about thinking, where are we going to be three months down the line, six months down the line, how do we get there? So I think that's important.

Leading for autonomy and accountability

[00:15:57] Vaseem: Surely that comes from the role of a leader, right? So if I'm a researcher, do I really care about three months down the line? I'm more focused on the here and now. So you're now in a leadership capacity: how do you manage that from a leadership capacity, to empower the team to work successfully within that team but also keep sight of the long-term goal?

[00:16:17] Om: I think for me it's very important for us to have a balanced team. You can't have just seniors or leads, and you can't just have juniors and interns, because it becomes lopsided. So I think that's why I create, at least in my case, this institution, and everybody is bought into it. My job is just to facilitate, so autonomy is important, accountability is important. So I make sure that every member of my team, irrespective of where they are in the chain, have their 10 percent capacity that they have to give to the institution building work, so they feel involved.

[00:16:51] Om: It's hard, because many times the critique I get is, hey, you are taking time away from the project. I'm like, don't look at the short term, it will reap rewards down the line. And to be honest with you, six months after we started our UXR department we pioneered eight new techniques, eight new methods, which were never used at Wildlife before.

[00:17:09] Alexandra: Wow. Also, something I've seen working well — I think it's important, as we're talking about leadership — is really inspiring people to go with you. For example, sometimes we're finding a new strategy, and really having people bought in: like, a few months, a few weeks later I was hearing everyone using words from that new vision, and really they were all feeling that they were contributing to it. I think if you can get to something like that, it's really, really impactful. And also a lot of communication, just bringing people along with you.

[00:17:43] Alexandra: And sometimes in the change management, also what I've seen working is kind of picking on their pains, kind of having people tell you the pains and then picking on that to explain the reorg and the changes, so it's not something that comes just top down, from like, oh, actually we, leadership, want to reorganize this like that, and everyone has to comply with it. It's also showing how that's going to have an impact for the company, but also how that's going to make people's life better as well, and how they're going to grow as practitioners, and how they can try out new things that have worked. And in this I also do a lot of storytelling. I usually tell my teams stories of how those new things that we're implementing worked in other companies, and how that's going to make us all create even better products and have even happier clients.

Communication at every level

[00:18:35] Vaseem: That's really interesting, and you mentioned the piece there about communication. As a researcher you're flooded with data points, so when you say communication, what sort of things are you communicating? Is it more sort of, right guys, this is our business priorities and this is what we're doing? Or what do you mean by communication?

[00:18:55] Alexandra: I was thinking, indeed — that's a great question — I was thinking of communication at all the levels. Communication on what you're trying to achieve, but also communication on why you're doing a certain thing, communication on what you are really trying to learn and why. I think even kind of helping teams be very intentional about what they're learning, because I've seen this, for example, in the decentralized setups: sometimes the product designers are not all super experienced and they just go and talk to clients, and you need to be there and be like, okay, what are you really trying to learn here? And then help really communicate around it: I'm trying to learn this, and then what I actually learned afterwards. So I think on all the levels communication is super important. And sometimes as leaders, if we feel we've said enough, it's never enough. You always need to keep repeating.

[00:19:44] Vaseem: That's really interesting. Duaa, I'd love to draw on your experience.

[00:19:47] Duaa: Yeah, I would say, from a researcher's perspective, as far as communication specifically in a decentralized model, there's the potential of your research only hitting sort of your direct stakeholders. And so there's always that challenge of making sure that you're having your share-outs with not just specifically your product teams, but making sure that — I heard the term earlier, "mini viral" — hoping that your decks go mini viral within the company as well, just to make sure that those findings are kind of being proliferated outside of just your direct stakeholders.

[00:20:22] Duaa: Because there's a lot of relevance, specifically when it comes to UX research that's more strategy based and more foundational rather than that tactical work, and making sure that that is being shared outside of just your direct stakeholders. So there's that aspect of communication when it comes to UX research, and I see that as being a little bit more challenging in a decentralized model, because generally your day-to-day is with your direct stakeholders.

Is there a best model?

[00:20:53] Vaseem: Yeah. I'd love, for all three of you, to get your perception or opinion on what is the best model. From what you're saying, it sounds like the larger the organization, a centralized model is perhaps the best way to empower researchers, but if you're a bit more nimble, at a startup, it sounds like a decentralized or hybrid model is perhaps the best way to empower researchers. So I'd love to open it up to you guys, because you're living and breathing this transformation.

[00:21:21] Alexandra: Wow, that's a hard question. I think I'm not sure there is a perfect model, because it depends so much on the industry and the context. But I tend to think that — I don't know — I still believe a lot in the power of product designers within teams. There is a point where it's not feasible to really have one in each team, so maybe at a certain point getting towards something mixed, I would say, when the companies grow. But it's hard to really say it's a recipe. It depends.

[00:21:51] Om: I think that's a very good point, it depends. Again, it's just not dependent upon the research and UX department, it also depends how the product teams are built, how big the product is, how many products are there. Because if it's a very big product, a lot of technicality, you don't want a floating resource, because the accountability can suffer: okay, I came in, I did my job, I don't have to take it to the end. So you need more accountability with an embedded resource, and maybe the cost justifies having that resource there. So I think it should depend on what's the need of the company.

[00:22:27] Om: I think both models can work, we've all probably handled that. But what I like about it is it's a learning curve. I think if you have a resource, for example a full stack product designer or UX designer, who has knowledge of — they can do some user research, they understand it, they can do UX, they can do workshopping, stakeholder communication, prototyping — nobody's saying be a master of all trades, but you can be a jack of all trades. That really helps, because you can pull in resources when you're falling short. But as products become more profitable and they grow in scale, I think it's time to build specialized teams. So you start going from this full stack model to specialized prototyping, specialized UX, specialized research, specialized market intelligence, and there's value there, but it has to be justified by the product.

[00:23:14] Vaseem: So moving towards more of an agile model. Yeah, got it, got it.

Stakeholders with their own agenda

[00:23:19] Vaseem: Another interesting thing that you mentioned there: you mentioned the word stakeholders, and I know certainly in your industry you've got quite opinionated IP stakeholders. This is what we want, this is what it needs to look like. And obviously then the role of the researcher is to go out and research, well, how viable is this, can we take this to market, what would it look like, what would that product need to look like? So when your researchers find something that perhaps doesn't marry up with what your stakeholders want, how do you manage that? What does that look like?

[00:23:51] Om: Yeah, I'll give you an example specific to one of the games I've worked with. We always have more than internal stakeholders. For example, if you're working for a franchise like Ice Age, 20th Century Fox has a stake in it. You are using their IP and it's a wrap around your game or product, so everything you're doing in the build has to be approved by them. And they have very different goals than internal stakeholders. For example, your business wants, I want to build a product at scale which monetizes, which will create a lot of engagement, and Fox is saying, yeah, that's all good, but make sure my IP shines, make sure it's true to the character. And they're right, we have to stick to the guidelines.

[00:24:28] Om: But here's the problem. When we launched the product they wanted IP fans to immediately recognize the storyline, so we had this first-time user experience onboarding where there was one and a half minutes of storyline onboarding the user. We saw 60% churn — I'm messing up the numbers, just to be clear, because of NDA — and we were like, this is not working, we have to cut the story short. But then Fox is like, no, but then we are losing out on the IP, I want to build that connection up.

[00:24:55] Om: So through research we did interviews with people who churned: why did you churn, at what point? And the non-IP players were like, we just came in for having fun, for the game, we don't really know this IP, and I don't want to spend the first one and a half minutes just listening to a story I don't understand. So bring those pain points, bring that churn data in front of them. We did some A/B testing and we saw, if we cut the story short by, let's say, 45 seconds, immediately the churn rate came down to 30%. So that's the value there, demonstrating, making sure that business goals are aligned.

[00:25:29] Vaseem: Yeah, got it, and obviously having the data to make those decisions and backing up what you're saying.

Empowering research at Aircall

[00:25:36] Vaseem: Alexandra, I'd love to hear your opinion, because I think I've mentioned the story of Aircall a few times, but I'd love to understand, from a research perspective, how are you empowering researchers, aligning with business goals? And I think the approach that you mentioned, teams were decentralized, so I'd love to understand a bit more.

[00:25:56] Alexandra: Yeah, I think it has multiple angles. For example, when I joined, not all the teams were doing research, so I kind of had to get more buy-in from the C-levels to it. So what I did is that I chose one or two projects where I knew I could buy time, because there was some basic logic that was already being built by the engineering team, so they had a few weeks of work before really heading for a clear direction for what we're going to do. So I kind of negotiated that, and then showed the value of what we learned: okay, this is not valid so we're not going to do that, this is what we're going to do.

[00:26:37] Alexandra: And then, when I talked to the CEO or to other stakeholders, I always use research quotes, and this is like a habit, so it doesn't come as just one argument that I pull off once, but it's like I constantly — well, constantly, not 10 times a day, but in our conversations — I bring on the users a lot, by like, oh, we actually heard this. And then of course there's a data part as well, as I was saying, and it's very important, but it's really by constantly doing this.

[00:27:07] Alexandra: And sometimes also, I think, even for the teams and the mindset, there are a lot of things linked to the empowering. For example, I do a lot of testing as well and lean experiments with the teams, and I remember one, like a terrible one, with one of my first teams that did this. I was like, okay, tell me what you've learned, and they told me all the things that they validated, and then I was like, okay, and what did you learn that didn't work? I realized that they only showed me the things that were validated, and then it was for me as a leader to keep bringing those things that were invalidated and be like, oh, we're actually not doing this, and we're not spending time doing something that's not worth it because we've learned this.

[00:27:49] Alexandra: I kept doing that, and in that case teams started changing their view on right, wrong, failure. And also they were more and more empowered, because I was like, okay, go out and find out if this works or not. And also leadership started seeing more and more the value of doing that, so I was able to hire product designers for every team, and then being able to do that more and more. But there's always, I think, this pressure still between, oh, we need to go fast and we know this works, and then being like, yeah, but this is still risky, it's still an assumption, and we can go fast, so we adapt and learn.

[00:28:30] Vaseem: Amazing. So what you're saying is, one, making sure the teams are agile, but empowering the teams to make their own decisions, use a bit of free will to explore different ideas, and I guess if you're going to fail, fail fast.

[00:28:44] Alexandra: Exactly.

Making insights actionable

[00:28:45] Vaseem: And Duaa, I know you've worked for the likes of Google and Lyft. I'd love to get your take.

[00:28:51] Duaa: Yeah, I think from the perspective of research, it's making sure that those insights are actionable. You come up with these insights after any sort of research project, and there's that stage of transitioning those insights into recommendations. I think most impactful projects consider the business constraints and the resourcing constraints within that stage of recommendations. So although you would want a huge change when it comes through the insights, you'll have to consider those business objectives and those resourcing constraints when you actually are putting together those recommendations.

[00:29:32] Duaa: And part of that is bringing along other stakeholders and other members of your team to kind of help you come up with those recommendations. It doesn't necessarily always have to come down to the researcher to come up with the recommendations. The insights, and the research, and all those steps to get to the insights are obviously the researcher's responsibility, but one thing that I think many researchers, at least successful researchers, realize, is that those recommendations don't necessarily have to be all on you.

Training, and change when teams get productized

[00:30:03] Vaseem: Yeah, really, really interesting. Another area I'd like to explore, if it's okay, is training. I think the whole theme thus far has been change management and managing changes as teams scale, and the role of researchers within those teams. So I guess, for you, as a leader, how are you understanding the needs of your researchers and understanding what training needs they have, so they're empowered to do their job?

[00:30:30] Om: Yeah, that's a good question. So I think, like everybody mentioned, autonomy is very important when it comes to even researchers and designers. We know that we have constraints, but we can't just keep them boxed. So definitely, as changes happen, people do question. I'll give you an example, I'll not say which company it happened at. I had a team who was reporting directly into me, I was the line manager, but then it got productized: okay, they are embedded now. The first thing was, the designers were like, oh, now we are reporting to a product manager, we lose the connection, we won't have a mentor, all that kind of stuff. My role itself changed in a way, because I was now supposed to go up another team.

[00:31:08] Om: But then the decision we took as a team — and again, as Alex said, it's not that decisions we are just taking on our own as leaders, you take it with the team. So for me, I use a lot of design thinking. All our processes come up, we come up together, it doesn't matter if it's leads or juniors, all of us together. So we recognize the pain points, we work on it. So we decided we still want to keep the craft side, it's fine if you have a direct line reporting manager, because everybody saw the advantage of having a craft, where they were learning, they were nurturing themselves.

[00:31:39] Om: So that's an example of how, with change management, you satisfy the team, and at the same time you understand why executives took that decision, because they were like, we want teams to be more accountable, and that can only happen if they're all sharing, breathing the same vision. And I'm like, that's well and good, but at the same time we are also an institution, we are a UXR human-centered design institution, we have our own values, and we want to make sure that even though each product team, each designer, knows what the product's vision is, they still understand what's the overarching vision for the department. So I think it's that satisfying, finding that balance between what stakeholders want and what your team wants, and fine-tuning it as you go along the way.

[00:32:19] Vaseem: Awesome. So it comes back to that central theme of constant communication, aligning to what the priorities are.

[00:32:26] Om: And yeah, even if you're decentralized, you don't want silos, because tomorrow, if one team is under, let's say, call it wartime — the product metrics are not doing well and there's more work than one person can do, it happens — if you don't have those connections with the department, who will come to help you, who will peer review? So it's very important for that reason.

Q&A

[00:32:50] Vaseem: Amazing. Alexandra, I'd love to hear your perception as well, but I'm conscious there's two minutes on the clock and I think we're going to open it up for Q&A. So I guess I'd love to open it up to the audience for a bit of Q&A to the panel, if that's okay, if there's any questions. Yeah, there's a box floating around.

[00:33:19] Audience: Hey, I'm just wondering if you're able to share any best practices of how to bring back qualitative and quantitative research and how they work together.

[00:33:33] Vaseem: Is the question around how to bring together qualitative and quantitative, how do you work together with the qualitative and quantitative research?

[00:33:43] Duaa: Yeah, I think the most successful researchers I've seen kind of use both. Mixed methods researchers, a lot of times you can start off with sort of the quantitative and then kind of dig deeper into the why with the qualitative research, and have different phases of research. And then there are times where there are very specific researchers who really focus in on the quantitative, and you're able to collaborate with their findings.

[00:34:13] Duaa: And I think another aspect that a lot of us kind of forget, a resource that we forget to really lean heavily upon, is the data science members within the company, and using that to supplement any qualitative work that you're working on as well. I think that strong connection between data science and UX research can really infuse that quantitative within qualitative research methodologies.

[00:34:43] Alexandra: Yeah, I've also seen this starting as well from data. Like, for example, I don't know, retention is dropping by X, or a certain user is dropping at a certain page, and then you need to understand the why, so you go more into the qualitative part. Or sometimes you do a wider discovery on a potential problem, you discover a problem, but then you need to know how much impact that's going to have, so then you try to get some more data on the segment or the volume or whatever the impact would be.

[00:35:11] Alexandra: So I think this usually needs to work hand in hand, even though sometimes it's like one and then backed by the other. But usually, in order to really have both sides of the medal, you need to check out both, and in time see how that evolves as well, because sometimes it's a surprise effect: you see some data, but then in time you learn that the trend is not exactly the same, and then you need more about the why it happens as well.

[00:35:41] Vaseem: Amazing. So I think we're actually out of time. There's some angry looking people over there saying we're out of time. So I want to thank you for your participation today, and hopefully you found the talk really interesting. Thank you.

[00:35:56] Om: Thank you, guys.