Standardising UX: A Roadmap for Success

Duaa Gettani at UXDX USA. Video: https://youtu.be/s1aCmhcIQFo

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Introduction and expectations

[00:00:02] Duaa: Hello. Great. My name is Duaa, and I know what you're all wondering: yes, like Dua Lipa, exactly like Dua Lipa. First of all, I want to give everybody here a round of applause for joining a session about standardizing UX research on a Wednesday after lunch. I know that doesn't sound like the most exciting topic, so you all deserve a round of applause for that effort.

[00:00:40] Just to go through things fairly quickly: my name is Duaa, and like I mentioned, I'm a researcher at Square, specifically on the Square Banking research team, so we're separated out. I have a background in academic research, and also a background in operations, specifically UX research operations, where I started at Lyft and then transitioned into being a researcher. So I have a bit of background in processes and standardizing things, and my experience will be coming from that perspective. And yes, again, I'm currently at Square, specifically Square Banking.

[00:01:20] Just to set expectations before we go through all this: there's no one-size-fits-all formula for developing and carrying out a research team, specifically within the context I'm coming from, which is fairly large teams. I know there are various contexts when it comes to research for folks. I'm coming to this from that context, and I just want to set those expectations. This deck is just meant as a starting point to help you begin to articulate the types of research and how to prioritize and standardize those research practices within your company.

Why talk to users

[00:02:00] Basic research, we know, is about talking to users, and everybody in the company should be empowered to talk to users. Within the Square Banking context, it's sellers, small business owners. Talking to users and conducting research is essential to creating products that meet everyone's needs and that have a greater chance of being adopted and of being a successful feature launch, or whatever it is. Ultimately, setting up those processes allows everyone to benefit from talking to a user and building that connection to the user.

[00:02:41] Peeling back the layers, the first basic layer, to simplify it, is talking to a user: that surface-level way to learn about products. Simply talk to a user about why they use the product, how they use the product, and who they are. That's the base layer that anybody at the company can do. The next important aspect is to understand a user's needs, goals and behaviors, to dig deeper. Specifically in Square Banking, we want to understand what sellers' pain points are and what their goals for their business are. We're talking to a lot of small businesses and a lot of small business owners.

[00:03:22] Ultimately, the sweet spot is to get to that center, where we're understanding users' pain points and addressing them and building empathy. The goal is to build that empathy for the seller, or for a user, depending on who your ultimate user is. That's the crux of it, where you truly want to get to. How do you do that well?

[00:03:45] Many of us have probably seen this framework before, but research can be engaged at any point in the product development cycle, from discovery to launch. For foundational research, we're generally learning about the people and the problem space. For generative research, we're focusing more on prioritizing and solving, and more on the solution space; at that stage you're usually in the define or refine area of product development. Ultimately, when we're thinking about launching a product, we're thinking about tactical research, the more evaluative research: does our solution solve the problem, and how could it be even better? Some researchers may not find that as exciting, but it's just as impactful.

[00:04:43] Research isn't meant to be a one-off solution. It's part of an overall process; as we see, we try to align it with product development, and ideally we incorporate research both when we don't know what to do and when we do know what to do. How do we start to instill those processes within a company so that research, and the impact of research, is more normalized?

Research team stages

[00:05:10] So, where you want to be and where it's OK to be, and again, this is coming from my perspective: where you want to be as a research team, and where it's OK for you to be in terms of structure. I'm simplifying this a little bit, but depending on your company, you might be the sole researcher, or you might be a designer doing research. That's the first stage. As things grow, and again this is a vague representation of my experiences in different companies, you may be a mid-sized team. You might have two to five researchers, but you're not fully embedded. You're still supporting various product teams, you're still trying to prioritize, you're still trying to figure out the best projects. And then the ideal scenario, which most researchers want, is to be a fully embedded researcher, with less context switching, where you're not switching from project to project and you're able to go with a project through its full development.

[00:06:14] Again, just to set a bit more expectation: there are challenges with each stage. Obviously a sole researcher would have very specific challenges as well, but there will be challenges in prioritizing and standardizing research at each of these stages. And it's out of scope for this presentation, so I won't be going too deep into methodology. This is just to give a fair framework of research and how it's structured.

[00:06:46] With a sole researcher, I think that's probably the most challenging stage, because part of it has to do with educating your team on research: what's the right type of research? Maybe folks reach out for research that isn't necessarily meant for UX research. So there's a level of education associated with being the sole researcher at a company, and with having impact to ultimately drive growth. The ultimate goal of the sole researcher is to show the impact of the research and potentially continue to increase headcount and grow the team. Part of that is prioritizing projects that will have that impact, and ultimately delivering that impact.

[00:07:32] Where we'll focus today is my experience as a researcher at Square Banking. We're roughly two to five researchers right now. There are challenges, but we do have a thriving research team, which is fantastic. We're also privileged enough to have an operations manager within the team, which is very essential for a research team, because there are a lot of logistics associated with research. I'm sure folks who have done their own research understand this: there's recruiting, and there are logistics associated with vendors. The challenges are quite significant if you don't have somebody helping with the logistics and operations side of things.

[00:08:18] At this stage you may or may not have an ops person, but luckily we do. The challenges here are about balancing and prioritizing the right projects, potentially consulting on other projects, and making sure that you're choosing the right projects for those full-fledged, full-service projects.

Research intake: roadmapping and the intake form

[00:08:41] Where do we start? The two ways that we intake research at Square are roadmapping and a research intake form. Roadmapping is usually the ideal state, and obviously that depends on how full-fledged the roadmapping process is with other folks at the company, and whether there's any ambiguity there. And then there's the research intake form, and we'll go a little deeper into that.

[00:09:21] With the research intake process, we obviously want to align the research roadmap with the product roadmap, and this is again not a one-size-fits-all formula. For us, it's quarterly. Folks could do it yearly if they feel the roadmap won't change as much, but we meet quarterly, and it will again depend on what stage of the product cycle you're in.

[00:09:51] At Square, our stakeholder alignment process entails either a workshop or a meeting where you get all of the stakeholders in the room, product management, design, whoever is involved, and we have everyone list out the potential projects that they'll have for that quarter. One of the benefits of doing it this way is that folks are able to hear what other product managers are working on, and they start to almost prioritize in their heads: "Hey, maybe that project's a little bit more important than the one I'm working on. Maybe the researcher's time is better spent that way." So it's pretty important to have everybody in the room when you're going through this process. Getting everybody in the room is a challenge, but especially when you're prioritizing and setting up your research roadmap, it's ideal to have everybody there.

[00:10:53] Looking at this sheet right here, this is the ideal output you would have from a roadmap session. You start to scope out the projects based on different aspects, and then this tab right here is what the ultimate roadmap ends up looking like. You have your time allotted, and you're able to show what you'll be working on in specific months. You're able to share this widely, so folks can see what types of work you're working on and understand why you may or may not be able to take on their project.

[00:11:25] Again, this is the ideal output you would get from one of these meetings, and I'll get into how to get to this point a little later. In addition, the other way to get research intake is through a research intake form. The main reason for this is that roadmaps change; this happens all the time. Priorities change; this happens all the time as well. So having this intake form available for folks to submit, instead of just pinging you, "Hey, I need help with research," means you're able to document it and understand what their needs are.

[00:12:11] Anybody can submit this form, but the way we have it, they're able to submit not only what the project is that they want to work on, but what the objectives are and what they would ideally get out of this potential project. So we have that documented, and we're able to prioritize ours as well. That, in addition to the roadmap, is how all the research is taken in.

Scoping and prioritizing projects

[00:12:45] What I just described is how you can begin to identify projects. How do you scope projects? That's the next challenge. These inputs are a bit of a balancing act. Whether the project came in during your roadmap process or through an intake submission, how do you ultimately scope out and prioritize the projects that you actually end up working on?

[00:13:12] First, you think about the purpose of the project. Consider whether this is tactical, low-level work, or strategic, higher-level, potentially more impactful work; where in the product development cycle this potential project is; how it will ultimately be used; and what kind of business decision it will be used for. Consider all of that in the purpose.

[00:13:41] Then think about company priority. Is this a high company priority project, or low? Where is the request coming from? Is it a higher-level request, or a more lateral request? Think about that when you're scoping and prioritizing projects.

[00:14:00] Timeline. This is extremely important. Researchers have limited bandwidth, depending on the types of research they're thinking about, so thinking about timeline is very essential. And not only timeline in terms of your bandwidth, but timeline in terms of when this potential decision needs to be made by. If you work on this project and the decision needs to be made yesterday, maybe this isn't the right project for you to spend your time on. Also consider the amount of time and resources the project might take away from other projects. Timeline is very essential when you're scoping and prioritizing these projects.

[00:14:43] In addition, the level of involvement. Research can be involved in a few different ways, which we'll go over. Is somebody else going to be leading this? Is this a project where we'll just be needed for review? How much involvement does research need in this particular project?

[00:15:03] And finally the priority matrix, which I'll also go over. All of these inputs ultimately contribute to the level of consideration. If something is high priority but lacks purpose, or the timeline doesn't match what you're thinking, that doesn't automatically mean the research should be a consideration. Just because it's high priority doesn't necessarily mean it's the right fit for research. You have to consider all of these aspects in the level of consideration when you're scoping out a project.

The priority matrix and levels of involvement

[00:15:41] I mentioned the priority matrix, and I'm sure folks have seen this particular matrix in the past. One thing I do is make sure that my stakeholders are involved in helping me decide whether a project is high risk and has low clarity. That's the point where research should be heavily involved. If a project is low risk and low clarity, research should still very much be involved, especially if there's low clarity. The other two quadrants are where you deprioritize research, especially if you're deciding between projects.

[00:16:19] Defining involvement can also be impacted by this priority matrix. What I mean by defining involvement is what I mentioned earlier: research could have a full-service level of involvement, or you could be just the guidance partner in a project, or you could be just consulting, with the stakeholder potentially doing the project on their own. What I mean by self-serve is that maybe a designer is leading the research and they just need review. Maybe it's a project with guidance and partnering: a researcher will advise on and approve the research plan and the questionnaire, making sure they're not asking biased questions and that the proper research techniques are taken into consideration. That's the guidance and partnering level of involvement.

[00:17:10] But when you're thinking about a full-service project, especially if you're a growing research team and you're thinking about how to grow the team and how much impact you want, the full-service project is the most important choice you make about what you're going to get involved in. You really have to think strategically and intentionally about the projects that you take on as full service. When we talk about full service, we're talking about fully taking on the project: recruitment, all the logistics, creating the discussion guide, creating the research plan, and then synthesizing and ultimately reporting out the research. Especially at Square, since we're not a fully embedded research team, it's definitely important that we're intentional about the full-service projects we take on.

In practice: upmarket sellers

[00:18:08] How does this work in practice? This particular project came up in our roadmapping process, so it didn't come in through an intake form. What we wanted to focus on was that multiple teams wanted to better understand the needs, perceptions and jobs to be done of upmarket sellers as they pertain to banking. We met with stakeholders, and since it came in through roadmapping and through multiple stakeholders, we knew it was a high-priority project, and that many people were thinking about it.

[00:18:52] Thinking about this framework for helping scope out projects: we knew the purpose was to help build strategy and continue to build out the roadmap for the future, so it's purposeful and intentional. For company priority, we knew it was a high-priority project, higher level in that decision-making process. The timeline matched up, and one important thing was that they brought this in during the roadmapping process, so we were able to make sure we had the bandwidth for it, rather than it coming in through an intake process.

[00:19:36] For level of involvement, based on the needs of the project, we knew we would have to have a higher level of involvement. And for the priority matrix, we knew this was high risk, and we also knew there was low clarity on what upmarket sellers need. So this hit each bucket when we were scoping, in terms of the level of prioritization it needed for each of these inputs. Ultimately we decided for this project to be prioritized as a full-service project. It would require full service because of the complexity and the level of logistics involved, so it was worth the researcher's time.

[00:20:28] To quickly go over the project itself, which has less to do with how we prioritize: it came in three phases. We started off by going through past research, creating a repository of past surveys on upmarket sellers and any interviews we had ever done with an upmarket seller. We created a whole repository of that past research. We then did internal interviews with account management, sales and folks who already talk to upmarket sellers, before actually talking to sellers, and we connected with data science and internal data. So we had this huge repository of all the information we already had before conducting any research. Then we presented this data in a research deck and did a company share-out, as this first phase.

[00:21:20] Then we gathered stakeholders for the next phase, the question prioritization workshop. We gathered stakeholders in the room and asked them what was left out of that first phase of research. We had them all write down any open questions they might have that weren't answered by this first phase. We created those questions and then prioritized them: if folks felt the questions others brought up were high priority, we prioritized those as the questions we would pursue in the next phase, which was the seller interviews. We ultimately interviewed upmarket sellers in remote interviews. This is just to give an idea of the phases of a full-service project that requires logistics and a lot of legwork.

Evaluative research and the research toolkit

[00:22:19] Lastly, some other practices to build out. I went through the roadmapping process and how to scope and prioritize these projects. I also went through a foundational project just before this, but evaluative research can be just as impactful and important. Especially in a team that you want to grow, where you want to catch the low-hanging fruit so you still have impact, those tactical and iterative projects can ultimately be a part of increasing headcount, because people see the value and the way that the research has had impact. You can be a major contributor to justifying an increase in headcount, and it's small and meaningful to have this space as a researcher. All this to say, we think about research having impact and value, and we sometimes make that synonymous with the full-fledged foundational projects, but evaluative projects can be just as important.

[00:23:34] This is just to give a quick understanding of a research toolkit. Like I mentioned, some mid-sized teams already have research ops, so they're able to help with creating best practices docs and a recruitment process. But when you're a mid-sized team, or even a sole researcher, it's all hands on deck. When you go through a process, you then create those best practices docs so the next person doesn't have to reinvent the wheel. That's the way we go through it at Square Banking. Again, research ops is very important here, but if there isn't a research ops person, there's always room to create those documents.

[00:24:20] Another very important aspect, especially if you have folks who aren't researchers doing research in your company, is to create templates, so folks have a starting point for creating research plans. One important one is a standardized survey question bank. We don't want folks reinventing the wheel when it comes to survey questions, so we have that template for folks to reference.

[00:24:42] Then the research plan and research brief are really important. The way I distinguish a research plan from a research brief: with a research brief, I'll usually have stakeholders create it first. They may have context that I don't have, so the research brief is a way for us to get a better understanding of the objectives of a project and what their existing assumptions or hypotheses are. Then I go ahead and create a research plan from that. That's my distinction between the research plan and research brief.

[00:25:17] A discussion guide template is also very important, so folks have a good template to use when creating their discussion guide for interviews, and they're asking the questions in an intuitive way that makes sense for approaching a user. Another really important template is our note-taker template and our note-taker sign-up sheet. The note-taker template is meant for folks who want to participate in sessions who may not be researchers and just want to hear from our users. You don't necessarily want many people in those sessions, so we have a note-taker sign-up sheet. That also helps folks be a part of the research process and be involved with the research from start to finish, so that's really integral.

[00:26:19] Finally, onboarding research tools and vendors is important as well, especially if you're not a full-fledged embedded team. There are times when there are important projects that you can't take on, so having vendors in your toolkit by onboarding them beforehand is integral. By vendors, I'm thinking about recruitment tools, because at times you might not be able to recruit internally, so you can pull out a recruitment vendor for potential research. Diary study platforms are fantastic. Survey tools. And then qual and quant partners to help with research that you may or may not be able to take on.

Research visibility and the embedded goal

[00:27:09] Finally, and this is very important, research visibility is integral to potentially having your team continue to grow. You can create that visibility through research decks and research readouts. Particularly at Square Banking, we have livestreams for our readouts, so anybody at the company can join, and that creates larger visibility and excitement around research. We also have a research newsletter, which is sent out quarterly, which allows folks to see the research that we've completed throughout the quarter.

[00:27:52] And finally, the most important is the research repository. I can't tell you how many times I get messaged about a particular question, or a particular area that folks want to conduct research in, that has already been researched. The research repository is fantastic to have as a reference for folks who may want to do research that's already been done. These aspects are great for evangelizing research and making sure we're advocating for research at the company.

[00:28:22] Ultimately, the end goal of doing all of this is to be a fully fledged embedded researcher. I don't know many teams that are; I think most researchers are prioritizing and supporting many different teams. But ideally your team grows, and instituting these practices lands you here, as a fully embedded researcher. Before we end things, I wanted to highlight some of the benefits of being a fully embedded researcher. You're able to actually be a part of building the product from start to finish, and you're able to re-reference your research instead of jumping from project to project. There's definitely less context switching, and as a researcher you're able to stay motivated, because in an embedded model you're able to make sure the research findings are translated effectively. So ideally, in the end, we have that fully fledged embedded researcher. Thank you.

Q&A

[00:29:32] Host: Thank you so much. Everything you said totally resonates, especially research repositories. I can't tell you how many times I've had research teams, or just anyone in the company, start a new research project, and it's, "Sorry, someone did the same thing a few months ago." And there have also been times I've brought research ops on just to get people doing it, and it's such a relief for the researchers. Amazing talk. We have a few minutes for questions, so I think you all know the drill.

[00:30:09] Audience: Hey. About the best practices docs: what type of information would that include?

[00:30:19] Duaa: Yeah. What I meant by that was just creating best practices. "This is the first time I'm writing this kind of survey; this is the first time I'm running a Kano survey. Let's create a best practices doc for that, for the next researcher who may want to run a similar survey," so they don't have to reinvent the wheel, like I mentioned. It could be any new project you're embarking on: document it as soon as you've gone through it, because all the time you spent figuring things out does not need to happen for the next person around.

[00:30:52] Audience: That makes sense.

[00:30:55] Audience: Hi, that was an amazing talk, thank you so much. For research teams that are just getting established or starting out, is there a tool that you would recommend for a research repository? We have research coming from a ton of different tools, and different people have access or don't have access, and we're really looking for a way to make it available to everyone.

[00:31:24] Duaa: There are definitely vendors you could reach out to that do that, but the simplest way is to just have a spreadsheet: this is the research, this is ultimately the impact, and the deck that we had for this research. Then plop it into a potential website that the company internally has access to. That's how we do it. And not only that, there are ways, even in Google Drive, to organize it so that everyone can have access. So there are ways to be creative if you don't want to use a vendor, but a spreadsheet is just fine to have as a link for folks to reference whenever they need.

[00:32:09] Audience: Awesome, thank you.

[00:32:13] Host: There's one right behind you.

[00:32:19] Audience: Oh wow, I like that one. I really liked how you distinguished between the consulting model versus self-service; you had three tiers. I was wondering if you could unpack that a little more. What really distinguishes guidance versus consulting and self-service? Guidance and partnership seem similar to me, but they're obviously distinct, so can you talk about that?

[00:32:50] Duaa: With self-service, I would say the researcher has very limited involvement. It's really reviewing docs, adding a few comments here and there, making sure nothing crazy is happening, because we've all seen some research that isn't ideal. So it's just making sure you have that visibility. With the consult and partnership level, research is still very much heavily involved. It's just that all of the logistics and all of the leadership in that project is not on research. You're partnering almost 50-50 with another stakeholder, whether that's design or the product manager. That level still has heavy research involvement; it's just not all on the researcher. Whereas full service is all on the researcher, but you carry other folks with you in the process.

[00:33:47] Audience: Really quick follow-up: do you define that as part of the intake process?

[00:33:54] Duaa: Yes. As part of the intake process, we have the level of involvement. For some of those really high-level strategic or high-priority projects, you're more likely to make sure that research is either guidance or full service, and definitely not self-serve.

[00:34:11] Host: We've got time for one more. I think I saw one right behind you.

[00:34:17] Audience: Hi, thanks for the lovely talk; it was very articulate. I'm curious how we draw healthy boundaries between what part of the research a researcher should be doing versus what a UX designer should be doing. Research is a big part of the UX process; it's a skill in the designer's toolkit, and of course it's a specialization for researchers. I noticed you mentioned self-serve and the different formats in your model. As a designer, how do I decide when I should be going to a researcher versus doing it myself? I'm more curious about the discovery phase.

[00:34:58] Duaa: That's a fantastic question, and something that's very important. Number one, the roadmapping process is when a researcher defines their level of involvement. So as a designer, it's important to get researchers involved in what you're thinking about for discovery as early on as possible, so you're able to know: do I have this researcher's time? Can they fully take on this project? When we're talking about the self-service model, a lot of the time that is distinctly a designer's... most of the time it's a designer who's doing it, because they don't have the researcher's bandwidth for more iterative types of research, and they just want a look at a few designs, and maybe it's not worth the researcher's time.

[00:35:49] Duaa: The short answer is that a researcher will give you that level of how much time they're going to be spending on it through the roadmapping process, or potentially the research intake process. So the level of involvement is kind of the onus on the researcher to explain to the designer. And even if the researcher is fully taking on the project, there's still a partnership with the designer: creating the designs, and making sure the designs are adequate for whatever research is going to be conducted. So there's still a level of partnership there.

[00:36:28] Host: We're right at time. Duaa, thank you so much. Amazing presentation.