Continuously Assessing Product/Market Fit

Nick Gill at UXDX Community: London. Video: https://youtu.be/jhH4eCkPB8E

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

The problem Flyt solves

[00:00:10] We work in hospitality technology, but that's not essentially the topic at all. To give you an idea of the problem that we're trying to solve, or one of the core problems that we solve: who here has ever ordered something from Just Eat, Deliveroo, who breeds [?]? So when you hit send on your order from your phone, that doesn't just magically appear in the restaurant. It goes to a tablet from Just Eat that sits next to the till, and someone has to go, okay, so there's an order, hit accept, copy that order out into the till. And there's a million other things going on around them. The kitchen might be on fire, the chef might be shouting at them, whatever. And that introduces human error. It also of course causes delay. You might not be able to accept it, you just get paid, your cancel [?].

[00:00:54] So the first thing that we did was to build integrations that meant technology can plug straight into the till. Pretty boring. There are hundreds of them [?], some of them are very archaic and difficult to deal with, and we've opened up the POS, the point of sale. We built our own apps, we've helped other people build apps, and now what we do, as we've pivoted, is to be a universal integration platform. So we open up a POS to just use an API, and then, hey, you can build whatever you want to do with it. That's the point.

[00:01:27] And the topic I was asked to speak about is problem definition, and that covers these areas, you can see it: qualifying the problem space, so insights, user journeys, jobs to be done and design sprints. I'm going to frame all of this within how we sprint right now. I say that with a caveat: this is absolutely not me dictating that this is a perfect model. It's definitely not perfect, and I'm going to finish the talk about why it's not perfect. So at the end I'd love to hear how anyone else has been having successful sprints, and just as importantly how you've been having bad sprints, and what your thoughts are on what we're doing right now.

Why we changed the classic sprint model

[00:02:06] So, problem definition is the title, and we look at problem definition in one week. Our sprints are a week long, we call them research week. We look to integrate better with development by supporting them with speed. A common problem of integrating UX is that, yeah, UX is great, it seems to make sense, but I want to get products out, you're in the way of my release cycle, you're a delay, and we're a barrier to products being made. If someone's not used to that then you're going to get a lot of pushback. We want to minimize lag from insights to scope, so when I'm creating insights we don't want that to sit anywhere, we want that to be straight into a scope that developers can actually work on. And something I'll come back to at the end is that we always try to keep research fresh. I think that's something that you probably hear from people in different ways, but we don't research for the gusto.

[00:03:00] So this is the classic sprint model. Who here works in a sprint format? That'll be cool. You're probably quite familiar with this, even if it came from the Google Ventures model that created the sprint, and it ran through these stages, and it took the loop away from expensive build time and risky launch time and just closed the loop from ideas to learning. The biggest problem with that is that at the same time they also just cut the developers out, so developers were waiting to build and launch and they just cut them away.

[00:03:31] And common problems with them: there's a long planning process which holds things up, teams still remain siloed, and you can still throw it in and over the brief. It's a time I quite like, but especially, you can run the best sprint possible, but if you're not involving the right people at the right time, then you're like me coming back saying, we did an amazing sprint and those are the races. And it still lands with the developers who have no idea what you've been doing, why we're doing it this way. The context is completely lost.

[00:03:59] So we try to sprint faster. It still takes a week, so it's not really faster, which is why I think I need to iterate on my titles. But we try to cover more ground in one week. We're a startup, about 30, 40 people, so for us we can kind of adapt our processes and play around with things, and what we want to cover is a problem to a deeper level inside of a week. And that took us away from the traditional Google Ventures sprint model. That took a little bit too much time. It dealt with it in one-week chunks and really assumes that you're going to be doing several one-week chunks. We try to keep it to one, two week chunks to have something that's usable, so that we can get that to developers as quickly as possible, so we're not holding things off and we're not seen as a bottleneck in the overall process.

Monday and Tuesday: scoping, experts and low-fi prototypes

[00:04:49] So to do that, stepping away from, hey, Monday is going to be all about the planning session, we do that Friday afternoon, probably with a beer in our hands, which makes the planning a real interesting one. And that's where we scope the problem. We want to understand where that brief's come from, why it's a problem, are we actually looking at solving the right problem, and getting our planning done, which involves whiteboards, sharpies and post-it notes. There's some of the restaurants [?].

[00:05:13] On Monday we like to use something. Something I really enjoy doing is our expert interviews. So internally we have a lot of people who are really good at their jobs, and we have people in data science, development, people who have built companies, run companies, people who know our customers better than we do. They're running our support desk and they have more context than I ever will. So whatever the problem is, I want to grab the person who's the expert in the room. We don't have to go and find them, we've already got people right in front of us who are really, really useful.

[00:05:44] This is also a great way of sneakily getting some key stakeholder buy-in. Our CTO is also one of our co-founders, so grabbing him for a bit of a technical point of view on a problem really kind of cements that you are going to be heading in the right direction with a project, and that you've got the backing of your top level.

[00:06:05] Then we build out user journeys from there. It's a nice shot of a spreadsheet. It's actually one of the best tools I've found for doing user journeys. I've messed around with loads of online whiteboarding tools and collaboration tools and they're really difficult, really clunky, and then I was sat in front of a spreadsheet in Google Sheets. You can collaborate with it, you can have multiple tabs to do different iterations, it's free, it's awesome. Now I'm back in love with Excel, and it's pretty.

[00:06:37] And Tuesday we move on to jobs, pains, gains, Crazy Eights and a low-fi prototype. Jobs, pains and gains: the J is jobs to be done, so we still use jobs to be done, which I really think is very, very valuable. This method of jobs, pains and gains I think is a really great way of clarifying your thinking around what you're about to embark on. So you run through your jobs to be done, you've identified your customers, you find out what they're hiring your product for, what are they trying to do. The example, the customer actually just wants a hole in the wall, is a perfect one.

[00:07:13] But then we look at the pains. So what's actually painful about completing that job? Is it time-consuming, is it difficult, is it the thing that they hate most in their week? And we look at what makes it hard to complete that job. Then we move through to gains. So what are the gain creators? Where are we already providing gains? Okay, so we know it's painful, it takes a lot of time, we've got something that kind of helps with that, soothing the pain a little bit, but what are the other gain creators that we could be doing that's going to make this better for them? What ones should we build, what ones could we build, what ones could happen, what ones is there no way we're going to build because solutions already exist? So jobs, pains and gains, if anyone's never touched on it before, is really useful and could be worth throwing into a sprint and playing around with.

[00:08:05] Crazy Eights is rapid idea exploration, progressing through concepts very rapidly. It keeps us fast. And then I like low-fi prototypes, so that might be post-its on a sheet of paper, or it might be some interesting wireframes, which I'm about to show you, in Sketch and thrown into Marvel, just to quickly work out, like, hey, we thought we were heading on the right track but have we? So I don't want to put this in front of users yet.

[00:08:33] A cardinal sin: Comic Sans, don't throw things at me. We use Comic Sans and real cheap, dirty wireframes at this level so that, one, it doesn't look real, because no one thinks you're going to build something with Comic Sans, which is great, but it's nice, it doesn't have to be pixel perfect, I can put it in front of customers quite quickly and we can play around with it. I don't have to worry about everything lining up. An iteration takes half an hour, and that gets us to prototype version two, which might look something like this, and it's clickable, it would actually let them play around.

Internal testing, then external testing

[00:09:10] And internal testing is our first benchmark. So again, we've got a lot of experts in the room. A big project that we've been working on at the moment, it's taken a couple of sprints, has been our own new developer portal, which doesn't exist yet, we've never had something like that. But I've got 25 developers around me. They're not necessarily the end user or the complete picture, but they are developers, they have used developer portals, they know the pains of building stuff, and I've got all of them around me. That's a brilliant resource.

[00:09:41] So I grab a developer from every team so that I'm not crippling their sprint, and take 20 or 25 minutes off each of them, run through internal testing, and they go, yeah, okay, now you've explained it I get what you're talking about, but I don't talk about it in this language, or that's just the wrong word. Which is brilliant. I want to get those kinds of problems out of the way as quickly as possible, because when I go for external testing, we've gone through all that effort of finding someone, dragging them into the office and speaking to them, and I don't want them to go, but that doesn't make sense, and then I get nothing else from them because they're just focused on this button not working. So internal tests, really useful. And you can buy someone lunch, buy them a coffee if they're really busy, get them to just ground you and check your work before you head on.

[00:10:30] Then we look at external tests. Another caveat there is that getting your external tests in earlier in the week means you've got to book them in earlier. We've been building our sort of beta testers program, where basically I track anyone that comes in and is willing to give me time and just say, hey, do you want updates on this, is it all right to contact you again and get you to come back, and keep sending you stuff? And they go, yeah, that sounds nice, and it's going on your mind map [?], wrap it up. So running a beta user program, there's a great reason why, and it makes that easier. You've got to be booking these guys in early, and inevitably some of them are going to spill over into Friday, and it's like, yeah, sorry, no. But we try and keep that tight to Thursday.

[00:11:16] Where we try and sprint faster with this is iterating on the fly. So we don't want to commit wholesale to changes because of one person, because something's not quite right for them, but if the first one has a complete shutdown, then, okay, cool, I'm going to tweak that in the half an hour gap before my next interview, and basically A/B test it and start to try and see throughout the day. If that change never comes up again then I think we're probably on the right track. So you started with version two, but by the end of the day you want to be at kind of version 2.5, 2.6, and you're already covering a lot of ground. You've already tested out my tweaks, the thing that you've tested with them, you've seen if someone's ideas grow onto that again, or if they're actually now just talking about the flow, which can be super useful. So that's where we're going to try and cover a bit more ground inside of the week and explore a problem to death, so that we can finish that week with some confident areas to talk about.

Friday: presenting findings to the whole company

[00:12:11] And then Friday morning, we consolidate all of our findings and put that into something where it's digestible, it's referenceable, so that we can go back to it, we know what we're looking at. We have a mega screen which is six screens stitched together, even people working remotely dial in, the whole company is there in some form, and we present our findings. We have an open forum, so every team will have a representative to talk about what they've done with that week, but any developer in the company, a developer who's not even going to touch the build of this product, or someone who I've not grabbed at any time during that process, gets a chance to give me feedback, give me criticisms, and have that open forum so we discuss ideas.

[00:12:56] And that's anyone from someone working on customer success, someone who's a data scientist, or a founder of the business, our CEO, to say, yeah, that's great, but I just spoke to Nando's and that won't work for them. I didn't do my job quite right, so now there's another area, another pocket of knowledge that I need to mine. And that keeps us grounded, that we're focused on the right thing, and makes sure that we haven't really messed up or missed anything important. I think Rory talked about something similar: when you're trying to move really fast, if you go wrong you're going to really go wrong very quickly. So we have to kind of have these points where we touch back to make sure that we haven't veered off and gone mental. Really useful.

Documenting publicly, feedback, and realistic goals

[00:13:35] Things that can help you sprint faster and keep things on track: documenting publicly and continuously. In the same way that your sprint's burning down day by day by day, we use Notion, if anyone's used that. We use Notion — not trying to sell you Notion, I don't represent them, but it's a really great product for us, and it's like a public internal wiki for the company. Everything is documented in different ways but in the same place, so anyone in the business at any point in the week can jump in and see where I'm up to, what the findings are and what we're talking about, which keeps us accountable.

[00:14:12] And it also means that it's easy to digest when we go back there. You can see, hey, we started with this idea but then we had to give up on it, why did we do that, why did we end up with a solution? Again, it's because we made this progress and that's where we ended up. Two: seeking out feedback continuously internally keeps us on our toes, but also means that we're going to slip up less often.

[00:14:31] And setting realistic goals. We want to sprint fast, but that doesn't mean, hey, let's try and do absolutely everything on the earth, everything associated with this problem, inside of a week. We want to do it really down to the minute level, as far as we can go inside that week, so that that problem is basically checked off when we move on to the next thing without worrying about it recurring. So make sure it's broken down into a small enough chunk. Don't look at it as, hey, we've got to build a developer portal, let's do it in a week. It's a great goal, but you'll still have a lot of blank spaces at the end of it.

Keeping research fresh, and owning it as a consultant

[00:15:05] Something I mentioned at the start I want to come back to: keeping research fresh. I think this is really important in how we serve our development teams, in delivering stuff to them as quickly as we can, and also delivering something of quality to them. So there's always going to be internal prioritization of projects. A question that we're starting to ask more and more is, yeah, that's great, someone's come to us with a problem, we're going to work on this, when can it get built? And that's about where your backlog is, whether teams are really running OKRs or whether they're just plowing on [?]. So we will prioritize projects that can get built sooner, so that it's not sitting on a stack gathering dust.

[00:15:43] I don't want my research to be more than a couple of weeks old, at most, if no one's going to touch it. If it's like, yeah, the team's going to build that in about two or three months, it's like, cool, I'm going to have lost all context myself, and if I did, they're definitely not going to have the full context. We're going to be sort of second guessing, we're going to be filling in the blanks, and we're making things up as we go as we're explaining it, because it's too old now. And I don't want to go to the wall fighting for a feature, fighting for a way of doing things, to say, you know, a customer definitely said this, when it's like a million miles away and nobody remembers. People want to be doing that as freshly as possible.

[00:16:21] So we do say no. Saying no is awesome, it's scary but it's good, especially when you just say no to research: put a pin in that, that's great, we'll explore it into the ground, but we're not going to commit wholeheartedly to it, because it's going to go really quickly and then go over there and no one's going to touch it.

[00:16:39] And something that I think is quite important is, I see my role internally as being a UX consultant. So I consult anyone in the business on any of my work. I own it for its entire life. If someone says, yeah, we're going to start working on the developer portal, cool, that's my baby, don't mess it up. So I'll sit in their planning meetings whenever needed, they can call on me to come and give them an hour, two hours, a whole day, whatever we need to do, so that the context is still understood and that we're working in partnership rather than [?].

[00:17:12] You're not always going to be able to involve everyone who needs to be there right at the start or throughout the process, but this is a nice way of nurturing and making sure that you're not like, yeah, research done, shove it away and I'm working on something else and someone's going to pick it up and it will just get built. I think that's a bad route to take. But being a consultant, I think, whatever your role is, is to say, okay, I've been involved with this and now I'm going to track it and stay with it, and development can say to you, actually, we've done the exploration, we can't build it in this way any more, that's going to change, now does this work? And I'm still acting as the voice of the customer, I'm still trying to represent our users to make sure that any compromises are still valid.

What isn't perfect about it

[00:17:53] So we're still iterating this process, it's still evolving, it's definitely not perfect. Working on the developer portal with this process works brilliantly for us. I think we had some amazing sprints for that and I was like, cool, this is like the sprint people want to happen, it worked really well for us. But it doesn't always work. You're not going to be able to just sit there and go, cool, I just ticked these boxes and then research have done, some awesome product's out.

[00:18:19] Time to adopt is always a thing. I'm lucky enough to work in a company where UX is pretty well understood and pretty well adopted, where I keep overhearing developers at work near me saying, no, we can't make that change, that's a really bad user experience, and it's like, I can quit now. That's just amazing to have. But when you're changing your processes to try to introduce something new, you're always going to have a struggle with that being adopted and embedded.

[00:18:49] And also, the teams own their own processes, they're evolving in their own way, and each has got its own quirks and its own way of doing things. So when we're pulling them in, we're working collaboratively with them, and we need to make sure that we're keeping up with them. So we're going to be changing, but I don't want to go away, come back and be like, here's the perfect UX process that exists, and they go, yeah cool, we're different now, that doesn't work. So it's kind of a never-ending cycle of iterating who we are and what we do.