The Role of Engineering in Customer Centric Product Teams

Adrian Trenaman at UXDX EMEA. Video: https://youtu.be/vXQnja5LqOU

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Meeting the people in the room

[00:00:00] Adrian: Thank you so much. Everybody can hear me? Okay, super great. Oh, thank you, I have a slide clicker. This session is not actually about me. It's all about all of you here. There's a well-known anti-pattern when we go to conferences: we end up sitting in a room looking at people or being told things, but not really getting out and meeting people and connecting. The idea of the Forum is that it's a chance for us all to connect and to share what we're doing with other people.

[00:00:37] We're going to use the AhaSlides voting mechanism, so the only thing I want you to do with your phone in this session is to have your AhaSlides open. I thought we'd do a systems check right now, so I want you to do an emoji check of your feelings right now. Let's see if we can... oh, this is great. Some surprise, some thumbs up. Thankfully nobody in tears.

[00:01:06] What we're going to do, just before we go into our context, is try the unthinkable. I'd like you all to do something that you might not normally do. I'd like you to turn around to somebody in the audience today that you do not know, and I want you to say: "Hello, my name is, here's where I work, and I am or I'm not going to the Horse Show House for a drink after the conference." You have 30 seconds. Go for it.

[00:01:49] Hi there, nice to meet you. I am going to go to the Horse Show House later on. Okay, fantastic, this is great. What have I begun? What I'm secretly hoping for is "And do you think, darling, if I hadn't gone to that Forum at UXDX, we would never have met?" That's the kind of thing I want two years from now.

Who is in the audience

[00:02:22] The first thing we're going to do is figure out who we are in the audience today. If you go back to AhaSlides and give us this view on who you are. We've got a nice spread here, and votes are still coming in. Product manager, one in eight. A PO, I thought it was a P0; a PO is a product officer. UX researcher, designer, and a reasonably high percentage here are developers. This is really great. And I've got 20% of people who are other.

[00:03:00] Now think about the context of why we're here today. There's sometimes this separation between the product designers and the UX researchers and the engineering teams that build the product. The thing is, engineers often have a very strong customer sensibility. We're trying to tap into that and see how we could bring engineers along the journey, maybe a little bit closer.

[00:03:36] I'm curious about the other, because I thought the first five groups would have more or less put everybody into a nice little category. Would anybody like to tell me what their other is? A few hands up. We have a hand up over here. Do you have a microphone? Oh great, awesome, thank you.

[00:04:01] Audience: Engineering management.

[00:04:01] Adrian: Ah, engineering management. Awesome, thank you for that clarification. Anybody else? Do we have any shocking others, like gymnast or florist, in the audience?

[00:04:17] Audience: Neither of those, but I'm an agile coach.

[00:04:19] Adrian: Oh, very interesting. We should talk SAFe later on, or even whether we should talk about SAFe. Anyone else want to throw in an esoteric role or party piece? Okay, let's keep moving then.

Where do solutions come from?

[00:04:42] Let's go to our next question. Let's start with this question: where do solutions come from? This answer could be different depending on the kind of organization that you work in. If you're a small group, ideas can come from anywhere. If you're a top-down organization, you might see an awful lot of ideas coming from the HiPPOs, the highly paid persons.

[00:05:18] What we're seeing is a really interesting spread here. We're seeing reasonably high that the ideas are coming from the business. We're seeing a good spread from the PM and product officer. I like it that there is 40% coming from the teams. I am a little bit worried, though, because it doesn't speak a lot for team autonomy. I'm a little bit surprised there. And then of course 10% from other. I'm always interested in the outliers. If you said other, would you like to tell us a little bit about where the other is coming from? Just put your hand up and we'll get you a mic out there. Great, thank you.

[00:06:15] Audience: They can come from everywhere: users, competitors, from your mom, I don't know.

[00:06:23] Adrian: Yeah, and that's interesting. Tell us more about that. Obviously they come from users, of course, and that could also be covered by teams using discovery. I'm interested in "from competitors". Tell me more about that.

[00:06:36] Audience: I think competitors have similar problems that you have, so they might already have solutions that you are looking for.

[00:06:47] Adrian: Yeah. And of course, we don't necessarily want to be copycats, but it's always good to be aware of what our competitors are doing. Any other thoughts there on where the ideas come from?

[00:07:03] Part of me is a little bit curious as well, particularly for people who are doing team-based discovery. You've got to love the vision around autonomous teams who get to learn about customer problems and then solve the problem, not in a top-down fashion but in a bottom-up fashion. I'm curious, though: does anybody have any examples of where they're doing this bottom-up work in a large company? Often large companies simply don't work that way. A lot of the strategy comes down from on top. Does anybody have any stories, any thoughts on how their teams are able to do this sort of bottom-up discovery?

[00:08:04] I love waiting. In my head I'm counting to seven very, very slowly. I know someone out there has got a little gem of information for us. Anyone biting? Yes, oh excellent.

Running interference on top-down decisions

[00:08:21] Audience: We did it within the BMW organization in the US, and we ran interference.

[00:08:29] Adrian: You ran interference. Who were you interfering with, and why?

[00:08:35] Audience: A lot of the decisions around building a product come from the executive level, even depending on the market. Our US stakeholder at BMW NA was intentionally trying to keep that from happening for the US market. They said one thing and we were doing another, where we were doing more discovery work, and then we built the product that way and it outperformed the global solution.

[00:09:02] Adrian: This is really cool. Just describe that for us. We've got senior management saying, "I want feature X, feature Y, feature Z," and then the team going, "Thanks very much, we'll do our own thing." Did you ever have that awful, awkward moment when they realized that you were doing something else?

[00:09:24] Audience: A little bit. There was an IT person that leaked that we were doing it a little bit differently, but it was too far down the road, thankfully. And we were doing a lot of upfront discovery work with dealerships and users, which doesn't always happen in the other ways that they do it.

[00:09:43] Adrian: That's right, and this is awesome. I have to say I think there are some cases where the executive group or the board are right. I think that happens every now and again, and typically it's when there's a very large strategic decision to be made that covers everybody. I sense, though, that companies work better when the senior people set the strategic goal, here's what we're trying to achieve, not the how, not the feature set. Then you can let the teams bubble up: "If that's what you're trying to achieve, here's what we think we should do." That sounds to me like how you did it. Anybody else working in that mix of top-down and bottom-up, or anybody wishing they were working in a company like that? We have another. Great, thank you.

Putting a stop on the problem statement

[00:10:47] Audience: Something similar to what was just described. What we found was that very often the solution was arriving with the problem. There was a problem, and if you looked at it quickly, an obvious solution. So there wasn't that discovery happening. There were no solution design sessions, no breakouts to say, let's come at this, think about it correctly and give it the time and space it needs to really get at the real issue at play here. What we implemented was a stop on the problem statement: give us the problem statement, stop, we'll come back to you. This is very new for us, but what it's allowing us to do is give us a little more breathing space. Then we can run a couple of solution sessions, see what kind of ideas get generated, and pick up the conversation better prepared. That's starting to generate good results for us.

[00:11:58] Adrian: I think that works well, because you've got smart people on the teams and they want to do the right thing. They want to do what they were hired to do. Did you ever run into difficulty with the senior people, maybe larger egos, larger ideas, or were you able to manage that?

[00:12:17] Audience: Not from an ego point of view, more from a time point of view. A lot of solutions are thought up on the spot. It's amazing how many solutions go into production that have been dreamed up that day. You find yourself in a sprint planning session and it's "How do we do this?" and whoever speaks first gets to do whatever the solution is. So it's a way of putting a stop on those shortcuts happening.

[00:12:51] Adrian: Absolutely. There's an interesting thing. There are a lot of product managers and a lot of researchers, and there are also a lot of people over here who haven't talked yet. We might come after you in a few minutes. I'm curious to know, in terms of where these ideas come from, do you find an anti-pattern happening where you do this amazing customer outreach or amazing customer research, and then you tell people about it and they do absolutely nothing? Has anybody run into that? I think it's a big anti-pattern. I'd love any stories around that. Oh, we have one over here. Great, thank you.

Developers building the better version anyway

[00:13:33] Audience: I work for an insurance company, and one of the big things we have is return users to our website. Many years ago there was a return user form designed in a PowerPoint slide.

[00:13:50] Adrian: PowerPoint-led development. Amazing.

[00:13:53] Audience: That was way before my time. But recently we've been working with the UX designers to do our competitive research and see what other competitors are doing. We came up with this whole business case and did a whole demo to people above us, and they were just like, "Oh yeah, the design looks good, but you don't really understand what the customer wants." So it was completely forgotten about. Then as developers we went ahead and built it, thinking, what's the worst that can happen? We could run this as a 1% test. It's not going fully into production.

[00:14:31] Adrian: So the better version coming from the developers. Fantastic. And I do think, in some of our applications at Google, we have a lot of traffic, and that means we can A/B test because you've got a lot of volume there. I'm often surprised at how the obvious solution that I would think works doesn't actually work at all. It's really amazing. It shows the value of getting up front with user research and testing and making sure that we get it right.

[00:15:05] That's great. It's interesting that so far some of the stories have been about development teams intentionally not doing what their strategy or their leaders told them, which I admire. Let's move on, because I think this maybe comes to the age-old battle between product management and engineering lead. What level of involvement does engineering have in solution development?

Engineers defining the solution

[00:15:53] Wow. I have to say I am really surprised at the way this one's coming out. I'm deeply surprised by this. I would have thought that maybe we would see a world where we've got a highly inspirational UXer or a product person who is calling the shots in terms of what the solution is. And I'm seeing here that not just do the engineering folk provide feedback, but the engineering team helped define the solution. I find that very interesting.

[00:16:37] Now I'm going to ask you to have a little think. If the engineers are the people who made the product, in a sense they have the cognitive model of the product but also of the underlying implementation. They know how things are made. They know the limitations. In some regards, if you are asking your engineers to define the solution, is that going to be a limiter? Are they going to limit the solution to the world that they know? The classic thing is that if all you have is hammers, then every problem looks like a nail. So is it that the engineering teams here just own all of the stuff and therefore they're highly involved, or are these engineering teams amazingly connected with the customer in some way? That's my question. I'd love some thoughts. This side of the room has been very quiet. Anyone there? Great, thank you.

[00:17:49] Audience: I go back to the dot-com bubble, when UX research was human factors, human-computer interaction, usability. My role was to define the user requirements and the user problems, prioritize those for the user, and then work with the engineer to figure out what was feasible and what wasn't, and to design an optimal solution within the business constraints. I think that's the right marriage: the engineers aren't saying what the problem is, but there is somebody who has talked to the user, knows something about the user and the environment, and is working with them to figure out the optimal solution.

[00:18:28] I also spent a number of years in the Department of Defense, where there were people who defined the requirements, and then the government contractors, Boeing, Bath Iron Works and so on, had their engineering teams work with human factors researchers to come up with the solution that would pass the testing required at the end. So I like these results, because you don't want somebody without a technical background coming up with a solution that's not feasible. But you want the engineers to understand the problem and the context of the problem and come up with an optimal solution for that.

[00:19:01] Adrian: Absolutely. We may come back there in terms of how we engage with the engineers so that they are capable of having that impact. But I'm also curious to know, and maybe I'm talking to a bunch of unicorns here, have you ever found engineering teams who really don't care? I'm getting big nods over here, so I'd love to hear your story if you can. Let this lady go first, maybe.

When engineering doesn't think about the customer

[00:19:35] Audience: I'm a bit afraid to say, but I will. We're developing a new product, and it's hardware. I do operations. The engineering team has developed a great hardware device that is not thinking about the customer. It's great, it's awesome, it looks nice, but it gives a lot of problems. We have addressed this with them, and they say the product is better and nicer. But it doesn't fit the other product needs, and they're like, "Okay, let's fix everything else, not the hardware. Speak to the customer, talk as much as you can with them and explain how it works." And it shouldn't be like that, it should be the other way, and it's just "no, no, no". So that's where we are.

[00:20:21] Adrian: Wow. Absolutely. We have another one behind you as well. Go first. Are you backing away? Okay, we have another hand up over here. Great, thank you.

[00:20:38] Audience: I think good solutions come at the intersection between roles. Engineers, from my point of view, especially in product companies that are trying to do some innovation, or especially in scale-ups and startups, need to be involved a bit earlier in the process than probably in the big corporate world where you have a lot of silos. I have worked as an engineering manager with teams of engineers that are very motivated to be involved in discovery at early stages, not just to understand the problem space, with product managers, designers, UXers and researchers working a lot with the problem space and interacting a bit more closely. So the solution is born at the intersection of these roles. It's not engineers coming up with it by themselves in a silo. It's mostly about collaboration, finding the great options and iterating a lot on those solutions together.

[00:21:36] Adrian: I love that, because I sense that you're right. But I've also been around since the dot-com, and I've been in many cases where the intersection point has been a fierce battleground, where you end up with a product manager or designer saying, "That is not your job. I am saying that this is the right thing," and an engineer going, "I don't care what you're saying, because I think that this is the right thing." I've seen it get physical. Has anyone witnessed this kind of tension? You still have the mic.

Mediating conflict through experimentation

[00:22:18] Audience: Yeah, I have been in a situation like that before in a bigger company. The way I have managed to mediate those kinds of situations is through experimentation. You have very opposite ideas here, or you really think that the way you want to do things should be done? Then let's run a two-week sprint, a spike or whatever you want to call it. Let's try something, let's prototype something, and let's put them face to face and see what's the better solution overall.

[00:22:46] Adrian: You mentioned a really interesting word there, which is mediation. It sounds to me that you have a pretty good sensibility around conflict management and conflict resolution. That's one of those soft skills everybody here needs to have, I think. We talked yesterday about the reasons why projects fail. A lot of the time it's people, people just not getting on. Any other thoughts on where maybe the conflict between, let's say, UX design and engineering has escalated, and how you've dealt with it? We've got one over here. Great.

[00:23:27] Audience: We have two different vendors. One works on the back-end side building the APIs, and the other team, from a different vendor, works on the front end doing the connection between the API. I had a few experiences in the company where I was literally mediating.

[00:23:48] Adrian: You are the marriage counselor.

[00:23:49] Audience: Exactly. And in the meantime, none of them did what we were asking for, and they were blaming each other. There was a huge conflict. They didn't talk, they didn't work together, but they were all part of the same scrum team. The solution I found was to bring them to the design thinking, to present the research, really bring them into the design process with us. Then they started understanding what needs to be done. The interesting thing is that they then started pointing out pain points for us that didn't come up in user testing or anywhere else, because of the way the solution architecture was built, the way they built the APIs. They started finding the gaps: "Oh, we need to fix this API, we need to fix this connection with the API." Now we really work very well together, they know their responsibilities, and they stopped fighting each other.

[00:24:44] Adrian: Job well done, because that's usually impossible with multiple vendors. Not impossible, incredibly hard. Well done. I find it interesting, because we talked about collaboration and the intersection between roles. If the friction is generating warmth rather than melting everything down, I think it can be very positive. But I do know that some organizations are very specialized and rigorous in their roles. They're very clear: a product manager does this, a UX researcher does this, and an engineer does something else. And that sometimes gets infused into their incentive systems, their bonus structures and so on. Is anybody working in an organization like that? These are typically larger organizations where the specialisms are very strong. Any examples there?

Getting engineers closer to customers

[00:25:54] No? This is interesting. It sounds to me, then, like you're all in a more fluid, less structured space, which I think is great. The next thing I'm looking at is this: as we generate the ideas, how do we get the engineering teams, who are usually asked to pump out code, engaged with the ideation or with the user? Let me give you two examples. In an e-commerce company I used to work for, every employee, all of the engineers included, got a credit worth maybe 50 a month to spend on the site. Basically this meant that everybody was eating their own dog food. If there was a problem at checkout or in returns, the engineers were all over it, because they wanted to be able to spend their credit.

[00:27:02] One other way to create great customer empathy, at the same company, which had a very strong customer focus: we had every engineer join the customer services group for a day and take customer service calls, actually listen to people. "I'm going to a wedding tomorrow and your stuff hasn't arrived. What am I going to do?" That's the kind of thing, and that creates amazing customer empathy. So from all of you here, what are ideas that you have used to get that connection with engineers, to get things more customer focused? Hello.

[00:27:48] Audience: I think if you have a good set of engineers already, you may be fortunate enough that they have good empathy. Going back to one of your earlier topics about the solution architects and the engineers, maybe I'm fortunate because I have a good group of engineers that I work with, but they have empathy because we've been dealing with a legacy product that hasn't worked really well, so we've had to fix those problems. Now we're into a new product. The solution architect said, "You're going to do it this way," and we pushed back and said, "No, we're not going to do it that way, because we know that won't work. We've been dealing with these issues for these customers, and we're not going to put in a solution that we know is going to be bad for the customers' experience." And, oh by the way, we're going to be the ones that get called in the middle of the night because it's not working properly.

[00:28:46] Adrian: Of course. Absolutely. Sorry, is my mic working? Okay, great, perfect. That's a really fabulous point. One of the things that I've learned along the way is that sometimes we get to work in a company where we are blessed and we don't know it. In other words, you end up with an incredible set of customer-focused engineers, everything just works and it's brilliant, and customers are protected. Sometimes, though, we work in a place where that connection isn't necessarily there. Great. Any other thoughts or ideas? We've got someone over here. Great, thank you.

Food rescue runs and how often to repeat them

[00:29:35] Audience: Thanks. I work for a nonprofit, so it's maybe a little easier to engage the engineers in that respect, because it's a charity. We do a food rescue project, so it's about food redistribution, getting food from retailers and restaurants out to charities. When anyone joins the company, you go out on one of these food rescue runs and you see the food tangibly going from one place to the other, because we do see that engineers are stuck behind their computers and they don't see it in real life, and it's hard to visualize. My question is, how often do you do this? An engineer might do that when they start at the company, but then we don't do it again. Should it be something that we're doing on a frequent basis, or when you get to a certain level of seniority in the company you go and do something else, almost like a reward for hanging on? They definitely lose sight of the purpose, I would imagine.

[00:30:47] Adrian: That's a great example. I used to come home all stressed out, and my wife would say, "Ade, you're selling handbags. Nobody's dying." But that's an incredible thing: the purpose is really clear. What's really interesting there is that moment. You could tell an engineer, "This is what's happening," you could show a picture, but it's the human connection when you take them there, when they actually see it, that's most lasting, I think. You mentioned frequency. I don't know, but I suspect quarterly is too much and maybe yearly would be good. Certainly if it only ever happens once, I think there's a half-life on that connection, and over time it's going to wear down. It has to happen more than once. But it's a great question. Great, this is super. Any more ideas? We've got one right behind the camera over there. Super, thank you.

Engineers in user interviews and kudos from users

[00:31:48] Audience: We've done a very nice thing with our engineers, which is involving them in the user interviews. For the internal product, they've been part of the interviews. That's one thing. The second thing is encouraging our operations teams, who are the users, to give kudos. If they like something, we say, "Hey, can you give a kudos to this person who worked on it," or share it on Slack, so they can also feel the love and feel the engagement. It works pretty well.

[00:32:16] Adrian: That's amazing. I'm curious, have you ever gone as far as having the engineers fully lead and orchestrate the customer outreach?

[00:32:25] Audience: No, not yet. It's always together with the PO, and they go together with the designers as well, so it's a squad working together. We also meet in person for a co-working event. We are a fully remote company, but we sometimes meet with diverse teams, operations, sales, engineers, product and design together in one space, just brainstorming ideas.

[00:32:52] Adrian: Great. I asked that question because I've seen, and maybe it's because I've been blessed with a particular group, a group of engineers fully arrange the customer outreach. What's nice about that is that the engineering team is there for the engineering team, as opposed to "We're here because we were told we have to go on the customer research the person has done." It's really nice to get that engagement. I love that. Before we go to the next thing, oh, we have another person over here. Thank you.

Three days in a taxi

[00:33:24] Audience: This is a story, but I'm not sure how practical it would be for all companies.

[00:33:28] Adrian: We love stories.

[00:33:30] Audience: One of the companies I worked in early in my career was a taxi app. We kept getting the same kind of issue or bug that we needed to fix, but there were only five or six developers working on it at that point. What our customer success team did was make us travel with the drivers for three days. We sat with the taxi drivers for three days. Each engineer had two drivers, and we spent the whole day with them. We got to see how they use the app and how it affects their personal lives. That gave us the importance of the things that we were trying to fix. I'm not sure how we can do this in bigger companies or with bigger products, but if you start this when you start the company, maybe that becomes a value of the company, and then everybody who comes in buys into the values and works according to that.

[00:34:20] Adrian: I think this is great. And here's the thing. If you can afford to do that kind of contact in a small company, you can definitely afford to do it in a larger company. It might be harder to arrange, but you can definitely do it. I often wonder: I've ordered a cab five times and they've canceled five times. Why are they doing that? It's not that there are five bad people. There's something else going on in the system that we need to understand, and you only get that when you really use the product.

[00:34:51] Audience: Exactly. One of the drivers kept losing the tip money that he was getting for the whole day. That was an issue in the app, but he thought he wasn't being good in his driving and so on. We could see in real life that he was losing it because of a bug. As a software engineer, we could see, "Oh, that's a bug that I have," and he's losing money for it.

[00:35:08] Adrian: Amazing. All right, we have time for maybe one more story. Let's have a look. Oh, super, again in the front. Super.

Stop being the hero

[00:35:15] Audience: Sorry, can you come closer? Sorry, I didn't know if I did right or not. Engineering deployed a new onboarding process for the customer, which was great, but the ordering part wasn't looked after. So I had to clean all the orders manually, and I also filed the bug saying this is not working well and I have a lot of work now. And they were like, "Oh no, you can manage that, there's no problem." So I said okay, I stopped cleaning the orders, and then on the email they sent about the launch, I replied and said, "Since this time the orders haven't gone. They are being created, but they're not leaving." And they were like, "Oh." I helped for a bit, but you created a problem for me and you didn't listen to me, and now I'm creating a new one. It's a shiny new onboarding process, but they didn't think about the other side. So I just thought, okay, I'm going to stop doing this and let it get bigger and bigger, and then it's, well, we're not shipping.

[00:36:18] Adrian: I think you may have done the right thing there. Just to recap, if I have it right: there was a problem, orders were coming in, but they needed to be fixed or cleaned before they could be processed, and you were busily trying to fix them on the fly. This is a great example of heroics, where one person saw the problem and went in to fix it. But heroes are a bad thing in organizations. You actually don't want them. What you want is people to say, "There's a problem here in the system, and I'm going to let it get bigger so that you'll actually fix it." Which of course I think was the right thing to do.

[00:36:59] Audience: Sorry, it took me a long time to learn the second part.

Should developers get involved in solution definition?

[00:37:04] Adrian: All right. Let's go to maybe... I think I can probably guess the answer to this one based on the mood of the room. Is it a good idea for developers to get involved with solution definition? Is that because one person voted? When you see 100% of anything you get a little bit worried. I've got to say I am really pleased with this. We're close to the end, but this is really interesting: not a single person here has said that it is a waste of time. Not a single person has said that.

[00:37:51] I think that speaks to the value of some of the discussion we're having today: engaging early, bringing the user research to the engineers, bringing the engineers to the users. This definitely has value. What's really amazing is that it's not just so that they can understand the context, but so that they can bring knowledge to the game and influence the decisions. This is great. We're just at time, so maybe we'll do one more emoji check, because I love emojis. Let's see if we can have a flutter of love hearts. I've almost got a flutter. There we go, this is fantastic. That's super. Listen, thank you so much, everybody, for joining today. I really enjoyed the conversation. Thank you.