How to Build an Engineering Culture of "Why"

Elizabeth Frekot at UXDX Community: Driving Success with UX, Generalist Design skills, and an Engineering Culture. Video: https://youtu.be/dcktqSTjnZM

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Welcome and the questions we're tackling

[00:00:00] Hello everyone, welcome to "How to Build an Engineering Culture of Why." You're probably wondering what's the UX researcher doing talking about this, but I'm here today as a result of last year's UXDX conference in New York. There were a lot of questions coming through from engineering managers, product managers, product teams across the board, and a colleague of mine, also a UX researcher, found that there were maybe some ways that research and UX research could impact this question. So thanks for joining me today everyone. If you're a designer, an engineer, a product person, welcome.

[00:00:41] Some of the questions that came up last year that we're going to look at today: we're going to start with a little bit of broader product intro and an intro to myself, but then we'll get into a case study of an example of a squad that I work with currently that does a really great job of building that culture of why around what they're building. One of the questions that came up last year during the conference was, how do we get engineers closer to the discovery phase, to the user needs and problems? And once we get them to that stage, how do we make it worth their time? It's not enough to get the buy-in for engineering teams and engineers to be involved at this stage, but how do we make sure that they're feeling empowered and confident to provide insights in that stage?

[00:01:27] And how does that all lead into this bigger question of getting the why behind what they're building? No one's out here to build a product... like we were talking about, the goal of UXDX is, there's bad software, you want to make sure that it's good software that people are actually using and they enjoy using. So these are the questions that we're tackling today with this case study, but I'd love to start with an introduction into who I am.

Who I am

[00:01:53] I'm Betsy, senior UX researcher at Ibotta. I wanted to share this little background just to give you a little bit more context on where I'm coming from when it comes to the product development space and the work that I do. I have a pretty strong background in psychology and psychological research. Throughout my entire academics I really focused on human behavior, feelings, and really synthesizing the full human aspect of things. So that's my academic background, really focused on people.

[00:02:23] I went into customer success and sales, directly talking to and consuming consumers, directly talking to potential clients and customers as well as current ones, really working on the customer mapping, journey mapping from a sales perspective, understanding pain points and how to fulfill those needs. But again, focusing on the human at the end of a service or a product.

[00:02:48] And now I'm here in UX design and UX research, which again is super human focused, very one-on-one, primary source. I live with the client thoughts and feedback in my brain in every meeting that I sit in. So I have the privilege of constantly being so close to the end consumer or the end user. In this case I'll be changing to "the end user" from now on. Throughout my entire academic career and my working career it's always very primary source data that's sitting in my head when I'm talking about solving problems or ideating around solutions. So just giving you a little context of the privilege I know I have to be so close to the end user, but that's not the case for everyone building a product.

The product triad and where research usually sits

[00:03:35] There's plenty of ways to look at a development team when building a product. Today I've just put this visual of the product triad, just to show visually a way to look at this development. We have product, we have design and we have engineering. Today we're talking about getting engineers closer to the why behind some of the solutions and technical solutions, because we're all aiming for this middle ground, which is solving real life user problems.

[00:04:07] But the problem here is that UX research typically tends to sit within only the design space. Maybe this intersection between product and design, where you're looking at real life client feedback or user feedback from a target market fit product lens, or from a user empathy lens from UX design. But there's all this visual space and conceptual space between some of the UX research and the engineering team where that overlap isn't so obvious to fill.

[00:04:41] Today we're going to go through a case study with one of the teams that I work with. Other researchers on this call know that we aren't typically embedded in one product triad or one squad at a time, so again, I have the privilege of being able to see a variety of different squads and the way that they function. Today's case study is focusing on a really positive squad culture around why, and the problems behind the solutions that they're building. This case study is really focusing on the ways that a particular squad that I work with is doing a great job of bringing their engineers closer to the end user and the solutions that they're building.

[00:05:19] A little bit of context about this particular squad: it is a B2B, business-to-business SaaS product. So some of the examples are going to be more B2B focused, but I will chime in with a little bit of direct consumer, D2C examples as well within this framework that we've been using. That's a little bit of the background. I won't be able to share too many specifics, so I have a video that won't have any visuals beyond sound, but it's still worth checking out.

The outcomes: engaged participants, empowered problem solvers, advocates

[00:05:52] I wanted to start this case study with the outcome first, which feels a bit backwards, but I think really shows why bringing engineers to the end users is so important. We talk about making sure we have a product that people will use, making sure that you're not building something as a Band-Aid fix. A lot of engineering squads might feel disconnected from that, but this particular squad has three things that I really think show the end behaviors of bringing them closer to the end user, and the benefits to the whole team when that happens.

[00:06:25] The first one is a bit self-serving and probably one of my favorite outcomes from this. When your engineers are really close to the end user, you've got really engaged and interactive participants. And I don't mean participants in an interview or a survey, but I mean participants in dialogue, readouts, workshops. You have a really engaged group of engineers who want to ask questions, they want to interact with the data and understand on a deeper level. As a UX researcher, I'm sure many on the call would understand when you're talking to the abyss, when you're just reading out a lot of findings and there are no questions and there isn't a lot of dialogue, it's really difficult to get to the meat of the insights. So again, for me this is self-serving, a really great outcome of having engineers that are close to the end users.

[00:07:20] But also for the team, you have empowered problem solvers. I think often we want to bring engineers into this discovery phase, but we're not really enabling them with the data to be confident and empowered to speak up. So if we're able to give them the context and the why — again we're back to that why of why we're looking at the particular problem, what's the analytical data, what's the user feedback — that's really grounding them in the problem that they're trying to solve for. So you have a team that's empowered to ideate, not just be in the room. That's answering that question of how do we make it worth their time: well, you want to make sure that they're prepared with all of the data and the feedback beforehand.

[00:08:08] And then finally you have advocates for data supported solutions. Again, they're feeling empowered. These two kind of go hand in hand. If they're feeling empowered, if they understand the data, they understand what the client end goals are, they understand the client problems, their day to day, the journey, they're able to advocate for data supported solutions. So we're bringing them into discovery, and we want to make sure that they're feeling like they understand the user enough to provide solutions that will matter, and getting to the root cause of things without just putting a Band-Aid on it. So that's a little bit of the outcomes from the work that we did with this team, that I will now go into in a little bit more detail.

Four ways we cultivated the culture: start with alignment

[00:08:53] But how did we get this squad to a point where they feel like they're user advocates, they feel empowered, they're engaged, they're excited? We had four actionable ways we cultivated this culture, so I'm going to jump into each of these four. The first one — got lovely hands stacked here for collaboration — the first thing that you need to do is find alignment across. We saw that lovely triad with engineering, product and design. We want to make sure that if this is something that's part... we want engineers closer to the end user, we want designers, we want product managers, we want everyone rooted in what is the end goal for users, what problems are they facing. Then it needs to be something that every team is aligned with.

[00:09:43] Another thing that was helpful in this particular case for this squad was bringing in additional squad leads. Traditionally the triad is the engineering managers, the product managers, the design managers, but this particular case study extended to UX research and product analytics. So finding collaboration and alignment with these included as well as squad leads helped create a space for me as a researcher, but also other people on the team, to understand the research in a deeper sense too. It created a space where new research or data analytics findings were shared across the domains a little bit easier than just doing a readout or sharing a report or something along those lines. You're already getting the leads across these different domains chatting about it, talking about next steps, thinking about ideation and solutions in that time.

[00:10:37] Additionally, it made scoping gaps in research, or questions along the lines from analytics as well, it made it easier to find the best equipped teams to understand how to answer outstanding questions. Oftentimes a product manager, an engineering manager, a designer, a UX researcher might have an assumption or a question that pops up, and the best case scenario to answer that question might be a user interview, it might be a beta test, it might be an A/B test, it might be something that analytics is best to answer with organic behavior, versus getting deeper into the why with some of the qualitative UXR methodologies. But having everyone in the room, one, allows you to scope who's best to answer those questions, and quickly gets the team aligned on who can support with that methodology too.

[00:11:29] When we think about beta testing and A/B testing, you're going to need designers to design the differences and you're going to need engineering, if you're doing a beta test where people are using real life code. You want buy-in from all the teams on what methodology is going to be best suited to answer that question. So having collaboration across the board was super helpful.

Ceremonies: the monthly empathy workshop

[00:11:53] The next thing that we did as a team was create ceremonies and carve out time for the engineering team to learn and engage with this information. The last one was getting leadership, leads, all bought into this idea that we want the end users at the forefront, but if it's something prioritized you have to set intentional time aside to make sure that the engineers are having time to think and engage and discuss the new findings as well.

[00:12:23] In this scenario, a lovely product manager created a once a month empathy workshop. This was something new to me, I'd never seen it before, and it really helped the engineering team get closer to the end users, through those who are speaking to our end users on a daily basis, or talking to or going through the processes that maybe they're not privy to otherwise.

[00:12:48] These empathy workshops broadened stakeholders across the company. So we had market research, UX research coming in doing readouts, discussion questions, giving a little bit more details, but we also had other stakeholders across the company come in and answer direct questions, talk about processes, talk about goals that they've seen. An example for the market and UX research teams: this particular group was newer formed, so starting in a green grass area of getting at foundational research, understanding the users before building a new product. So market research was actually brought in, in alignment with myself, to talk personas, to talk higher level goals of our users, and really get into the nitty-gritty foundational work.

[00:13:39] I've added customer care here as an example of someone to bring into an empathy workshop. Maybe you're more in direct consumer products where you're getting a bunch of customer feedback through chats in-app, or you're getting emails, you're having complaints or something along those lines, where they're able to speak to that and really bring more light into the engineers, into understanding what those problems are, coming more organically through customer care.

[00:14:05] And then finally, with our B2B process, we had a lot of account team interactions. Sales teams really getting into the nitty-gritty of what they're hearing from our customer and user base, that could bring more context to our engineers as well. So this was something that really creates the ceremony of every month making sure that we're bringing in somebody new, and the product manager really prioritized this, which you could see made a big difference.

Pulling engineers into live research

[00:14:35] The next one is pulling engineers into live research. This especially came up from our conference last year at UXDX: what are some of the small things, maybe you can't carve out all the time, you're still working on the buy-in to get that extra hour a month or two hours a month for more ceremonial user empathy workshops. You can always pull in engineers for a live interview or a live usability test.

[00:15:03] I'm thinking about when I'm invited to a pre-recorded webinar versus a real time live one. How engaged are you? The difference between watching someone talk or having the access to ask questions in live time are two different kinds of engagement styles. So bringing them into the live sessions helps not only the engineers but also the researchers.

[00:15:27] In my previous experience, having the engineers in some of these usability calls, or in these live call interviews, helped get some direct questions back and forth. With the SaaS product they were questions that were technological capabilities, and questions that might be something I would have to ask engineering after the fact to get more context on. But having them already in that interview helped me ask those questions up front without the delay back and forth, and be able to assist this user in real time.

[00:16:01] Additionally, engineers get primary source feedback. Again, back to the "hey, watch this five minute video" or "watch this highlight," "watch this 30 minute call" — getting that experience of watching people use your product in real time just provides additional context, and might increase engagement as well.

Celebrating and sharing user wins

[00:16:23] And finally, this one is my favorite one and I think probably the most popular too. If you have a smaller research team, or again have a little less buy-in for some of the more spread out empathy workshop type styles, we have celebrating and sharing user wins.

[00:16:42] No one wants to build something that no one uses, and surely don't want to make someone's experience worse. And this is where UX research gets to shine. We have that primary source data. Say they couldn't make the live interviews, but maybe you're interviewing 20 people and you want to give some of that feedback. Maybe you heard something really cool about a problem that everyone was working on for weeks, and you hear somebody with an audible gasp and an excited demeanor. Hearing that in real time, that's the best. Hearing that in a video short clip on Slack, also really great. Seeing it as a quote in a recap deck, not as exciting.

[00:17:27] In this scenario we're going to pretend that you're an engineer that worked on a feature for the last six months, and Betsy, your UX researcher, just interviewed somebody who just interacted with this new feature. Again, I couldn't show you the feature in particular, so you get to just see my awkward interview face, but imagine this option in real life versus just a quote. So this is feedback that if you're an engineer that you've been working for six months...

[00:18:02] "Ooh, oh my gosh, that is such a good idea. So good." Okay, so imagine you've really worked on this super hard and you have someone saying "wow, what a great idea." The audible gasp sits in your head a little bit longer than just a quote — "wow what a great idea" — versus a real person interacting with it. So in this case I'll often share maybe some feedback recaps, a short highlight, or sprinkle in some quotes. Sometimes we get from our account teams, we'll sprinkle in some feedback that they're hearing, but really encouraging that process as a product progresses.

[00:18:41] Speaking of progress, also UX benchmarking has been something that's been super helpful with really being able to communicate user wins with engineers. Like I said, this was a pretty new product, and being able to benchmark this new product versus an experience previously, and then show that over time, that with every feedback session we had a few features pop up or a few problems pop up, we did some ideation, we pulled them in, we looked at technical solutions. At the time of the next benchmark, seeing that number go up and that problem be solved is just a nice satisfying moment for everybody. And sharing that win directly with the engineers who worked on it for months has been really helpful and super exciting for me as well, because you get to win with everyone.

Collaboration is what holds it all together

[00:19:30] But all of these lovely actionable ways to cultivate why would fall flat, you could have none of them, if all the teams don't collaborate from the forefront. So if we look back at that product triad, if you have buy-in only from a few members and not everyone, you have engineers looking to get into some real-time feedback from clients, but maybe you have a researcher who's uncomfortable bringing them into live interviews. But maybe there's a different solution that they can bring, to bring the engineers closer. Or if you have a researcher who's still so excited to bring this to the engineering team, but there might be some problems with getting the buy-in from engineering and product. Just working together to make sure that everybody is on the same page when it comes to keeping the end user at the forefront is the biggest takeaway from this case study: that all the product leads had buy-in from each other and also from leadership, that it was worth the engineers' time. And those end behaviors were the result of that.

[00:20:35] So thank you so much everyone, I would like to open the time for any questions.

Q&A

[00:20:43] Host: Excellent, thank you very much Betsy, that was really, really interesting, a very thorough walkthrough of the different ways to get involved. I'm a massive fan, I'm a convert already, of getting more people involved in upfront research, because I think all the points that you said, as well as just making sure that everybody really internalizes what are the needs and what our customers actually care about, and what they don't care about too much. So yeah, if anybody has any questions — Hope's doing a great job of saying how great the event is, so that's obviously a kudos to you Betsy — but if anybody has any questions, please do post them in and I'll ask them through to Betsy. We have a few minutes of time.

[00:21:28] Host: I'm going to kick it off. You mentioned about the empathy workshop, and then the next one was, "I don't like watching things pre-recorded, I much prefer things that are live." So how valuable are those empathy workshops, considering all of the information that's going to be shared is secondhand? It's not going to be direct from customers, it's going to be via somebody else.

[00:21:53] Betsy: Yeah, that's a good question. I think the benefit of the empathy workshops is, even if the data is something that maybe happened previously, you're getting that real time live conversation. If you're just reading a report and you have questions, maybe you're putting it in the comments and then maybe I'll get to it. But if we're having a workshop and I'm explaining a persona or a user group and you have questions in that moment, like "okay, what does this mean a little bit more," or "can you show me an example of this," you have that real time. The data is not in real time, but the conversation is in more of a workshop style, rather than "here, read this, if you have questions let me know." I think that can be a little bit intimidating, for people to just reach out to me. "If you have any questions" normally goes unanswered. Or if I'm going to be honest, when you see those emails or Slack messages come in, you don't read them.

[00:22:49] Host: No, or you just emoji it.

[00:22:51] Betsy: Yeah, yeah. "I'll get back to that, I will, I promise."

[00:22:56] Host: Sounds so cool, love it. So you mentioned about the challenges of getting people involved in live participation, and that could be either the engineers aren't interested, or the researcher's a bit protective, or different things like that. What have you seen — I guess it's a two-parter — in your experience, where has been the biggest challenge, and then two, how have you overcome that?

[00:23:18] Betsy: Yeah. Funny enough, I'm thinking about the engineers I work with now, and the desire to be in the interviews is definitely not the problem. That could be your problem, it's not a problem I've been facing. I think when it comes to... I think scheduling, so just finding the time. Sometimes they're hours, sometimes they're 30 minutes. There's definitely — I don't think it's the engineering teams or even the researchers, it's just aligning that schedule. And you know, if you're interviewing one person, you probably shouldn't have four or five people, you don't want it to feel like a panel. So I think it comes more down to a logistical "can we make that time," and where can we find the most value of spending that time in a user engaged interview or something like that.

[00:24:09] Host: So yeah, often it's about having a good ratio of observer to participant, and then getting that scheduled. If they have a lot of feature work, or they're working on a lot of things at the time, they might not have 30 minutes to an hour to spare. And do you ever have a situation where it's recorded and people are just sitting in the back room watching, or is it always that panel? Because I agree, a panel of people can be quite intimidating.

[00:24:37] Betsy: Yeah, there's definitely softwares out there that allow you to have the backend observers. That's where, in this case scenario with SaaS products, you want to make sure that you're getting people who are using your products, so we're kind of sitting in the Zoom world and Zoom doesn't have the back end. But yeah, definitely trying to lean on some of those products where you can pull in observers on the back end has been really helpful, when I was previously more on a D2C side where we had maybe participant panels we can pull from.

[00:25:10] Host: Awesome. We have a question from the audience. Can you give some tips on how to build that collaboration across teams when everyone is remote, will never be in the same space because people are in different countries?

[00:25:21] Betsy: That is a great question, and I'm currently very remote, so that's perfect. I think the first thing is getting comfortable in a shared chat, and building trust. There's plenty of books about teamwork and building that trust first, but you want to feel trusted, and you want to be able to trust everyone that you're working with in the collaboration space. So building trust, whether it be finding that time to do more in-depth intros, but building that you can trust them to do their side of the work and find that middle ground.

[00:26:04] Betsy: From a remote standpoint, I can only speak from experience with this team, but I do think if you have something like Slack, or if you're on Windows, just really getting into finding shared wins. It's really nice to be able to bond over that, so that's something that I found to be really helpful, like "hey, let's all be really excited about this audible gasp." And from there it just kind of makes... it's a small thing, but those really make a difference. So I think if you're looking at remote, just really celebrating the wins together has been really helpful on our team.

[00:26:36] Host: Awesome. One thing you touched on that got me thinking was, it's building trust, but when you mentioned about the engineers and "oh, this is a technical answer" — is there any kind of training or anything, I guess media training, that kind of thing, of "you can say this to customers and you can't say this to customers"? Something like that, because people might unintentionally — I don't think it's a negative thing — but they might unintentionally say things in a way, or divulge information that they shouldn't.

[00:27:09] Betsy: 100%. In that scenario that I was speaking to, they were kind of like my ghost writer, hilariously enough. They were just sending me direct messages and I was then sending it to the user, and I would change the vocab to user language. So I think when it comes to the training, that's something that you would breach once you get there, but in that scenario it was more like they were note taking and sending me DMs. And so then you have — again, that's kind of why I started with my intro, always kind of been front people facing — so you have to lean on people's strengths. If you feel like you have a technical answer and don't know how to reach this person, reach a client, you have to lean into a skill set that maybe your designer or researcher might have, to get that communication across. So in that case scenario, I translated.

[00:28:02] Host: Excellent. And have you had that, I guess just following up, have you had that situation where you do have that panel, the physical panel of people, and is that something that you've encountered? I know with other people they do a little bit of a user research 101 training.

[00:28:22] Betsy: Yeah. Well, the first thing I always have people do is change their names to "notetaker." I've never had more than two at a time, and primarily I like to do one, because I have a lot of one-on-one interviews. So it'll be my note taker and then myself. And the 101 that I give is, "hey, you're a notetaker. Also if you do have a question, communicate it through me. If you have a comment or you have a follow-up, I'm very open to having the DMs coming through." I know that that's not the case for everyone, I want to just say that, because I kind of do about 10 things at once and that helps me stay very focused. So when it comes to the 101, it's more so just being observant, and rather staying more of an observer.

[00:29:16] Host: Brilliant. That actually brings us to time, so I just want to thank you once again Betsy for sharing, that was really interesting.

[00:29:24] Betsy: Thank you.