Listening For The Signal Through The Noise

Andrew Carey at UXDX EMEA. Video: https://youtu.be/0QzqePHEPPE

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Too many stakeholders, too much data

[00:00:04] Hi guys, my name's Andrew, and I'm going to tell you today how Realife Tech made personalisation a core part of our product, and give you some practical tips on how to find the data that you need to pay attention to when working on your own products. First, just a little bit about me. I'm a user interface and user experience designer, originally from the Bahamas, and I've been designing stuff for about 20 years. I'm currently the Head of Design for Realife Tech in London, and we help some of the world's largest brands, sports teams and venues connect with their fans through personalised, responsive experiences.

[00:00:39] Personalisation is an incredibly powerful user experience. Big brands like Spotify, Google and Amazon use it every day to keep us glued to their products. The way that Realife Tech arrived at personalisation was organic, but also hard fought.

[00:00:59] A few years ago, we were engaged in multiple discovery processes and interviews with key clients, one of whom was a well-known London football club with a new stadium that was about to open. Over the course of six months, we spent time with every stakeholder in the club. Many people from many departments told us what they wanted to do and who they wanted to cater for. They had lots of user personas that we could use, but every department had different personas and needs for the types of users that they specifically cared about.

[00:01:32] Marketing wanted to make sure that their promotions and products were shown to the right people. The transport team wanted to make sure it was easy to get to and from the stadium during match days, but on non-match days there were different requirements. The content team wanted to make sure that the news, scores and other important information was always up to date and region specific. The tech team wanted to use single sign-on to identify their users across all their different channels. The ticketing team wanted to make sure that tickets inside the app were easy to get to on match days, but on non-match days were safely tucked away. And the website wanted to be able to update their website and app in one single process.

[00:02:09] How could we cater for all these different stakeholders and all these different users at the same time? After a lot of time researching and collecting data, the sheer amount of information was quite overwhelming, but a pattern was starting to emerge.

[00:02:29] It's easy for product processes to overlook the role of customer insights to inform them. Using just pure research to evaluate an idea can cause problems late in the innovation process, and there's a sunk cost fallacy that can make it hard to back out and start again. Customer insight and context need to be part of the process from the beginning. Research and data can indicate a behaviour or need, but data alone isn't enough.

[00:02:58] Lots of data can sometimes be lots of noise. But how do you find the signal within all the noise? Human-centred design requires an understanding of the people that you are trying to reach, so that you can design from their perspective. Not keeping this in mind can lead you to create something because you can, rather than because it's useful.

Keep everyone involved, keep an open mind, kill your darlings

[00:03:23] Our product team needed to advance the product, and we needed to show value to our clients. And because the design team was heavily involved in this process, we were able to contribute meaningfully in providing stakeholders with the user experience they hadn't considered in all of their requests. And this would ultimately be more beneficial than the things they thought they needed.

[00:03:46] Here's how we did it. Keep everyone involved. I can't stress this enough: the best way to break down barriers is to not have any. Everyone in the product team brought their own unique experience and expertise to the group. Engineering could help assess the technical viability and complexity. Product managers could make sure it was captured, categorised and checked against existing features. Customer success could give us valuable insights into the nuances of the client relationship and give important context on their requests. Project managers kept us focused and on track. And of course, designers were synthesising all these inputs and advocating for the end user, and also reminding everyone that design is not just about how something looks, but also how it works.

[00:04:39] Also, keep an open mind. Don't be super specific in your interviews. Influence comes from a huge range of directions, and unless you keep your mind open, you're never going to go to unexpected places. Context is everything when trying to understand stakeholders and users. At our football client, we interviewed as many stakeholders in each of their departments as possible. We asked them generalised questions about what the experience was they wanted to provide and who those experiences would best serve. Having this standard set of questions helped us to unpack, categorise and compare the answers later.

[00:05:13] When we started looking at possible solutions, there were many ways we could go, and our trove of interview answers really helped us get in the stakeholders' shoes and recognise the differences between them all. Having so many different points of view really helped us to look at ideas from different angles, and organise brainstorming sessions with stakeholders where we could exchange ideas and try new things.

[00:05:38] Be prepared to kill your darlings, guys. I tell this to my design team all the time. Don't let your biases or your personal preferences keep you from reaching the ultimate solution. Having strong opinions is great and preferable, but don't hold on to them too tight. Hold them loosely. Be willing to change your mind when presented with helpful information that counters your own belief. This will serve you well in product design and in life.

[00:06:05] In the early stages of our project, it was a lot easier to stay detached and iterate, and as time goes on this becomes harder, because of many factors, including being in love with your own ideas. Our first proposal for the solution for our client was ultimately rejected. And after some soul searching, we realised that we had put a lot of effort into putting something forward that would be really good for us and our platform, but wasn't going to meet our client and their users. It was humbling, and honestly slightly embarrassing, to be given this news from stakeholders that we were working so closely with. We tried selling the clients our own self-serving ideas, and now we needed to let go of that and serve the users. We swallowed our pride and put the users above everything else.

Designing for the future, and improvising as you go

[00:06:52] Designing for the future is hard, especially when you are also designing for the present. And unfortunately, stakeholders can't always tell you what they want tomorrow. However, if you keep at it, you can pick up valuable clues on where you can evolve. This will let you design something with a solid idea of how it can evolve over time.

[00:07:11] Our stakeholders were asking us a lot of different things, and we were realising that we couldn't please all of them individually. So instead of catering for just the stakeholders, we focused really hard on the users, and found that if we built a flexible system with some sort of governance, we could achieve our goal and enable the stakeholders to achieve theirs. When coming up with this new solution, after we had proposed a [climber][?], we came up with a system of governance for the product that would start off with just a few simple rules. And we knew over time we could add new rules when new projects required, without rethinking everything every time we had a new requirement to cater for.

[00:07:51] Improvisation and adaptation and the ability to overcome are really important. The data you need to execute your solution is going to yo-yo over time. You're going to need qualitative insights at the beginning, but as time goes on you'll need smaller bouts of interviews and research, then move quickly to test, refine and optimise, so you don't paralyse the whole process. Making a lot of micro-decisions was easier than making huge sweeping ones.

[00:08:21] The data and the work we were now doing was working well, but we topped up on details with key clients and internal stakeholders to make sure we were solving the right problems at the right times. With so many stakeholders involved, we needed to be nimble: moving quickly, adapting to requests and validating ideas with key stakeholders at every step of the way, so we wouldn't get derailed. Project management helped schedule twice-weekly catch-ups, and once we got into a rhythm, it was easy to iterate, present and check our ideas with everyone involved. The frequency of these catch-ups also provided good motivation to quickly iterate ahead of the next call and not get too attached to your ideas.

A personalisation engine built on context

[00:09:00] Using those ideas is really how we made it work. Our sports and entertainment apps aren't in constant use like Instagram or TikTok or Facebook. Our users engage with our products during key touchpoints and while doing some real-life activity, and we wanted to be able to respond to them better. We needed to use our product to make sure that we catered for their needs, and personalisation was really going to enable that.

[00:09:27] Because we kept an open mind, we were able to see that the only way to success was to be really flexible: to build a system that can adapt to different contexts and serve different audiences within those contexts. We made every corner of our app dynamic, so it could be updated by anyone at any time. We also introduced this idea of context that would govern all the dynamic content and show the right thing to the right person at the right time. This system was built in a way that we can add to it over time, because we had one eye on the future, and this system was built to change over time.

[00:10:02] This enabled us to make this a cornerstone of our user experience, and we're still building on top of this personalisation engine today. This isn't exactly what our stakeholders were asking for, but what we delivered was a solution that would enable many other future solutions: something that was above and beyond expectations, and something that would be a unique selling point for our products.

[00:10:29] So to wrap things up. Keep an open mind. The answers you are looking for are everywhere, and keeping an open mind gives you a better chance of spotting them and acting on them. Kill your darlings, guys. Have strong but loosely held opinions. Have one eye in the present, one eye in the future. Leave hooks in your ideas for future improvements. The present you can help the future you a lot more than you think.

[00:10:59] Improvise, adapt and overcome. Don't try to do too much all at once. Short, focused bursts help you iterate quickly and understand the value of your ideas. And while all of this is going on, keep everyone involved. Use the strengths of each discipline involved in product decisions to your advantage, and keep communication open, honest and constructive. Well, that's it for me for now. I hope you enjoyed this short talk and can't wait to see what you do next with your own products.