What Winning the Lottery Taught Me About Team Preparedness and Research Continuity

Madeline E. Smith at UXDX Community: Luck, Trust & Design: Building Resilient Teams & Secure Interfaces. Video: https://youtu.be/ryP8SKJX2lQ

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Are you prepared for the unexpected?

[00:00:07] Madeline: Thank you so much, Rory. Like Rory mentioned, my name is Madeline Smith. I am a researcher at Cisco currently, and I'm talking to you today about winning the lottery. My question, before I get into this: are you prepared for the unexpected? When we talk about the unexpected, you might be thinking of things like your emergency preparedness kit. Do you have a go bag in your car that's ready to go for whatever emergency strikes? But that's not really what I'm talking about here, although it's probably a good idea to be ready for some things like that. I mean in the context of work. What would happen to your work if you suddenly went away?

[00:00:48] Think about your calendar and what you have scheduled for this afternoon. What would happen if the power went out and you couldn't join your meetings? What work do you have planned for tomorrow? What would happen if you needed to call in sick? How would your team members handle your absence? What if you were absent longer, not just for an afternoon or a day? What if it was some sort of indefinite absence?

[00:01:13] There are lots and lots of reasons that people disappear from work unexpectedly. We often hear about the hit-by-a-bus scenario, but that's pretty dark. People told me to take it out of this stream, so I'm going to reframe it in a more positive way. We're talking about winning the lottery. What would happen if tonight you win the lottery, all of a sudden you're a billionaire, and you never need to work ever again? What would happen to your work? Is your team prepared to continue the work that you've been doing without you?

The time I won the lottery

[00:01:45] I wanted to share a little story about the time that I won the lottery. Well, not exactly; we'll get there. One year ago, in April 2024, I had been working on a long-term project for about a year and a half. I was really proud of the work that the team had done so far, and I was super excited about the momentum we had going. We had just gotten some great new team members onboarded. There was a lot of excitement and a lot of interest in moving this project forward, and that was so exciting. I loved these people I was working with, and I was super interested and invested in where we were going.

[00:02:23] I was planning for an important research study. I was in the middle of recruiting participants who were going to be participating in that research. This study was really crucial to help us plan what we were going to be doing next for the rest of the project. I went away for a planned vacation for a couple of days, and then right before I was supposed to come back to work, I got a call that completely changed my life, and this is where I won the metaphorical lottery. I got a call from the adoption agency that I had been working with, saying that my husband and I had been matched with a baby.

[00:02:57] We got this call Monday night, and Wednesday morning we went and met our baby. So within a day and a half, I went from planning to go back to work to suddenly going on parental leave, and that meant that I was away from work for several months. I logged in that Monday in this state of confusion where I was panicking and trying to figure out what I was doing. Do I have a crib? Where was the baby going to sleep at night? What did I need to buy for my baby? What am I going to name my baby? All of these kinds of things were going through my mind, and work was still something I was thinking about, but not too much. I really wanted to be able to turn that piece off and move on to this next exciting chapter of my life.

[00:03:42] I was able to send a quick message to my boss and let him know what was going on, and log into work for just a couple of hours to send out some messages and let everybody know. A couple of people on my team knew that I was in this process, that I was waiting, and that I might suddenly get a call like this. Other people I hadn't shared this detail with, so some of my collaborators were expecting me to come back to work the very next day, and all of a sudden I wasn't there.

[00:04:12] I set everything up and then I logged off for months, and in my absence, my team was able to pick up the pieces, continue the research study that I had planned, and keep the project moving along. When I logged back in several months later, with a four-month-old baby, I was able to catch up on all the amazing work that they had done. The project had moved forward, and all of these dreams I had for this really big project I was so excited about had come true. The team had done amazing work even in my absence.

[00:04:49] How did the team continue my work when I suddenly went away? The answer here is pretty simple: they were prepared because of the detailed research plans that I put together.

Research in academia versus UX

[00:05:01] I want to do another story time here, about how I learned to do these really detailed research plans. The answer to that is that I'm a recovering academic. I spent about 10 years in academia, first as a grad student, then as a professor, and that's where I learned how to do user research. The focus of research as an academic is on generating scientific knowledge and publishing, so the research process looks a little bit different from what we're used to here in industry and in corporate UX research. I want to walk you through a little bit of that process and talk about what's similar and what's different.

[00:05:42] The first part in UX research is having some sort of idea. You're going to be identifying your research objective, choosing a research method, and getting ready to do your study. The next step in our UX world is buy-in. We need to convince our stakeholders of our proposed approach. This might be a quick meeting with them. This might be a PowerPoint deck you put together or a Word doc you write. It might just be a quick Slack message saying, "Hey, here's what I'm planning." But you've got to get your team on board.

[00:06:18] Then you might have to do some sort of review. This really depends company to company, team to team. Sometimes there's a group of people who need to review your survey or your plan before you go out and contact users. Other times you don't even need this step at all. The fourth step is actually conducting the research itself: going out and doing your study, analyzing data, and eventually presenting your findings. This is the UX process that you're all pretty familiar with.

[00:06:47] I mentioned that the academic process is similar but much more convoluted, so let me walk you through that. In the academic world, again, we have that same kind of idea. We need to identify a research objective and choose a research method. But this can take a really long time in academia, because it's not just based on some things you're observing with your users or your data or conversations with your stakeholders. We're looking through academic literature. We're conducting literature reviews. We're reading papers. We're discussing and debating ideas, collaborating, all these different kinds of things, so it can take a long time to even just come up with the initial research question that you want to ask.

[00:07:32] Getting buy-in in academia is really, really complicated. That's unlike my experience as a corporate UX researcher, where my salary is pretty set, I know what I'm going to be working on, and in general my team's on board with that. As an academic, things are a lot more tenuous. You need to apply for and get your own funding to be able to do that research work. That might be applying for internal or external grants to fund your work, and this process can take months or even years and involves a really lengthy, complicated process with a lot of documentation that you have to put together.

[00:08:14] Similarly, the review process in the academic world is really, really onerous. You need to apply for something called approval from the institutional review board, or IRB. The IRB is an organization that's looking out for the protection of human subjects, so people who participate in research. They have a really strict set of rules that researchers need to follow before you can have any sort of contact with your users. This means you need to put together an incredibly detailed research plan and send it off for their review. They're going to go through it with a fine-tooth comb to make sure it matches every single requirement that the team has.

[00:08:56] After you've gone through that, there are usually a few iterations and revisions back and forth. I had times where I had to revise and resubmit my IRB approval just because of a typo in one of my survey questions, or something wasn't exactly grammatically correct. So every single piece of your work needs to be pre-planned and reviewed really carefully by yourself, the people working on the project, and then this external board. And again, after several months of waiting, eventually you get approval to go ahead and do your research, which might look exactly the same as the kind of research that we do here in the UX world. You might be doing interviews or surveys or whatever other types of research methods.

[00:09:43] You go out and do your research, and you analyze your data. In the UX world, we're often presenting our findings internally to our stakeholders at a meeting. In the academic world, we were writing research articles and publishing those papers at conferences, in journals, books, things like that. So the steps are similar. We still go through these four phases of the research process in both the UX and academic worlds. But in academic research, this is really, really slow and really, really detailed. There's so much red tape that you need to go through at every single step to make sure that you're meeting all these requirements.

[00:10:21] Like I mentioned, this is where I learned to do research. This is how I was trained to be a researcher. I was used to having to get all of my ducks in a row, having to write out every single thing that could possibly happen during a study, and having that really carefully reviewed by lots of people before I could even start talking to folks. So when I first came to the UX research world, I was a little bit surprised that people didn't spend as much time doing these really detailed plans that I was used to seeing in my previous environment.

[00:10:57] When I would talk to people, they would say, "Oh, no, I'm too busy. I don't have time for that. We're just going to do this study right now. We're going to start our survey. We're going to launch it tomorrow." I've been in UX for about four years now since I made the transition, and over time I've learned that it is really valuable for me to have a detailed plan. Not exactly to the level that I had as an academic, and not spending months and months to create it, but I do still take a lot more time and put a lot more thought and effort into my detailed research plans than a lot of my colleagues are used to doing.

The research plan template

[00:11:35] So what does my research plan framework look like? There are a number of things that I always include in my research plan, and I've got a template document that I'm happy to share with folks after this. I'll share a link at the end of the talk. My template includes a couple of different sections. The first thing is a project overview. Then there's the section that is the research plan itself: that includes what research methods we're doing, what participants are involved, what the deliverables are, the timeline, and a RACI matrix. I always include links to other files and documents and things that are relevant to this project. And then I include some number of appendices, depending on what is relevant for that particular project. That might be recruitment messaging, participant screeners, the interview guide, the survey questionnaire, or the test plan if you're using an unmoderated platform.

[00:12:34] In my experience, a lot of my colleagues are used to writing out some of these things, creating interview guides, writing their survey, but a lot of folks don't take the time to put this all together in one single document. And sometimes when I do this... my template here, if you look at it, is nine pages just of the outline, without actually filling in the content. So these can be really long documents once I fill in all of this information, and that can feel pretty overwhelming, both to create and for stakeholders to look at sometimes. But I'm here to convince you why it's important, so let's walk through it step by step.

[00:13:17] The first section is the project overview. I like to use a consistent title in my templates: whatever the project is, include that this is the research plan. I like to include the date that this research is going to be starting, usually with the year and the month. I always include some status of this document. Is this the draft? Has this been approved? When was it last updated, and by whom? And then I have a bulleted list at the top that calls out some key information. These fields might change a little bit depending on the study and the information that's required; for different projects we need slightly different information. But the idea here is that anyone who opens this document could skim the first couple of sections, this bulleted list right here, have a sense of what the project was all about, and be able to figure out whether it's something they need to look at more or not.

[00:14:18] Excuse me. The next section is the research plan. It feels a little silly to have a research plan section within my research plan document, but that's really what it is. This is the section that gets into the details of what the research is going to be. I have a section that's usually about a paragraph long where I'm setting the context for this project and outlining the objectives of this particular study. The academic version of this would have been a 10-page literature review, an introduction to the research problem, and a long introduction to the objectives of the study. We're not doing anything like that here. This is about a paragraph long.

[00:15:03] For the research method, again, this is a fairly short description of whatever methods or methodologies are going to be used for this particular study. Similarly, there's a section on the participants. Who are the people that are going to be included in this study? How are you recruiting them? How many people are you trying to talk to? What is the recruitment approach that you're going to be using? And then the expected outputs. What types of deliverables will you be creating from this study, and how are you going to be sharing them? Is it a PowerPoint report? Is it a meeting? Whatever else it might be.

Timeline, RACI and links

[00:15:43] Really importantly here is the timeline: from when you're kicking off the project, to when you expect the research plan to be agreed upon by all the stakeholders, when participant recruiting is happening, when the research sessions themselves will be happening, data analysis, reporting, creating your draft, and eventually scheduling your research readout. Whether or not people are writing this out in a research plan, they're going through all of these steps. You're usually having some sort of readout at the end of your study. You are usually creating some sort of report. You have to do some analysis to get there. People just don't always take the time to write out what this timeline is.

[00:16:27] For me personally, that's incredibly useful. I am a deadline-driven person. I need to know when something is due to be able to work backwards and make sure that I'm keeping myself on schedule and going to meet that deadline. So having this in writing is incredibly valuable, and so is being able to review it with my stakeholders during a meeting before we actually start the project, to make sure we're all in alignment: what are we doing, when are we doing it, and does this meet the project's needs?

[00:16:59] OK. Another piece that I like to include here is the responsibilities matrix, or RACI matrix. Who is responsible for, accountable for, consulted on, or informed about the different sections of this plan? In the projects that I'm working on, often that's myself as the researcher or any other researchers I'm working with. That might be folks from the product management team or design team that I'm working with. There are developers involved, or leaders, or change management, or content, lots of other teams who might be involved as well. It's really helpful for me to have everyone on the same page and really clear about what it is that we're doing and how everyone is involved.

[00:17:44] Another section I always include in these documents is links. I also had a couple of these up in the summary at the beginning, but this section of my document can sometimes have 10 different links, depending on how many different tools you're using for the project. I always include the shared folder that we're all working from, and the chat space, if there's one we're using to talk about this project. If there's a Miro board or a Dovetail project, OneNote, Figma, PowerPoint, whatever files we're working with, I want to make sure that the links are here and easily accessible.

[00:18:19] My goal here is to make sure that anyone who looks at this research plan can pick it up and find the other information that they need. Similarly, when I do my research reports, my final readouts, my PowerPoint template also has a link back to most of these files or to this document, so that, again, people can see how the project started and where it ended up.

The appendices

[00:18:46] All of this information is somewhat common for people to include in their research plans. What I don't see a lot of are the appendices that I include. I include one to three appendices, depending on what the project actually needs. The one that I almost always include is recruitment communications. For me, writing the messages that I'm going to send out to participants or potential participants, inviting them to participate in my study, takes a surprising amount of time: carefully crafting a message that is going to grab their attention and really draw them in to participate. Having this outlined, and I usually use these table formats where I can just copy and paste it into an email, makes it so much easier for me to make sure that I'm using consistent messaging from one study to the next, and to get alignment with my stakeholders so everyone knows what messages I'm going to be sending out.

[00:19:47] Similarly, if I'm using some sort of screener survey to screen potential participants before they schedule, I might include those details in an appendix as well. Particularly if I'm using a platform with a panel to recruit participants, I might specify the different demographics and requirements of who is included and who is not eligible for this study. I might include the specific questions they're going to be asked on that screener: something that is a single question where you're either accepted or rejected, or something that really has multiple options they can choose from. Just detailing this information about what I'm planning to do makes it so much easier, so that everyone knows what's happening.

[00:20:33] Then there's the interview guide. Everyone I know who does interviews creates some sort of interview guide as part of their research process. It's not all that common, in my experience, for folks to include it in the same document as their research plan. But I've found that to be incredibly valuable: just having one link that I can share with anyone working on the project, and they have all the information they need. My interview guides are in a format similar to things you've probably seen before. I have an agenda listing the different sections of the interview. For each one of those sections, I include some information about the targeted time I'm planning to spend on it, and maybe a note to explain to the facilitator what this section is all about.

[00:21:18] I include my questions and some notes to self: pause for questions, start recording. And then the questions that I'm going to be asking, along with optional follow-up questions depending on how the interview goes. So again, you've seen the idea of an interview guide before. You might not have thought to include it in your research plan. Similarly, if you're running an unmoderated study, I include the test plan in the same document, word for word what is going to show up on the testing platform that I'm using. And again, with a survey, same concept: I'm including all the details that are going to be in my survey in my research plan.

Go slow to go fast

[00:22:01] Like I showed you, this is a really long document. The template itself, with all of these blanks, was about nine pages. Some of these sections don't appear in every study, but the research plans that I work with tend to be quite long. The project I'm working on right now has about a 14- or 15-page document. These are living documents that I'm constantly referring back to as I go through my project. But a lot of people hear "14-page document" and think, I don't have time for that. Why would you ever take the time to put all this information here? That's a question that I've gotten a lot.

[00:22:43] My answer for you is that the benefit here is going slow so that you can go fast. What do I mean by that? I mean that taking the time up front, at the start of your project, to fill this out and document all of the details that you have planned will make the rest of the project go so much faster. When I start working with a new product team on a new long-term project, one of the first things I do is take a copy of my standard template and start filling it in for this project. I fill in most of the details, and then I highlight the sections that are going to change from study to study.

[00:23:24] I'm doing this while I'm doing my introduction meetings with stakeholders, stakeholder interviews, planning, putting together my roadmap, all the things that you're doing as a researcher or as a UX team member at the start of a project. You can be working on this document as you go. It isn't taking a lot of extra time; it's really just helping you document what you've already done. Then when it's time to run my first research study, I don't have to scramble to pull something together, because I already have it prepared. I have my basic outline, and I can go in and change whatever pieces I need to.

[00:24:01] Similarly, once I finish that study and I'm ready to run the next one, I can just copy and paste that document, make the changes that I need, and get ready to go with my next study without having to spend time thinking about all these questions that I've already put thought into before. So while it might slow you down the first time you're filling it out, in my experience it really does speed things up for the follow-up studies that come from there. Similarly, having these documents allows anyone else to pick up your work and continue your project, like I shared from my experience.

[00:24:39] Thank you all for listening to me. This was my talk about what winning the lottery taught me about team preparedness and research continuity, and I hope you're all convinced that writing detailed research plans is a good use of your time, even if it means going slower to be able to go fast later on. Again, I'm Madeline. I'm a lead UX researcher at Cisco, a recovering academic and a parent by adoption. If anyone is interested in seeing that research plan template, feel free to send me a message on LinkedIn or shoot me an email, and I will share the link with you. Thank you.

Q&A

[00:25:11] Rory: Thanks very much, Madeline. That was really interesting, and I guess congratulations on winning the lottery.

[00:25:18] Madeline: Thank you.

[00:25:18] Rory: I'd say that's quite a shock, going from... normally people have a nine-month lead-in.

[00:25:28] Madeline: Yeah.

[00:25:29] Rory: Brilliant. We had a comment from Amy and Brian: they love the comms section being part of the plan. It tends to be an afterthought. And Brian's saying "Yay." So some people agreeing with you there. It's probably just a little bit tight for questions; we've only got about 30 seconds left. If anybody has any questions, please write them in and we'll share them with Madeline afterwards. Other than that, I just want to thank you once again, Madeline, and I hope everybody out there enjoyed it as much as I did. Thank you.