On The Road To Dual-Track

Flavia Neves at UXDX Community: Eastern Europe. Video: https://youtu.be/Mn3RLMQ2HBM

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Introduction

[00:00:03] Hello everyone, welcome to my journey of implementing dual track at FREE NOW. I'd love to be one of those people who jump on stage and don't need any sort of introduction, but the reality is most of you don't know me, so here it goes. I have spent the last 15 years working mostly in the startup world. Started in Portugal, moved to Ireland, then worked in Silicon Valley, and eventually landed in Barcelona, where I remotely helped companies worldwide with their product and growth challenges.

[00:00:36] In the last 18 months I've been working at FREE NOW. FREE NOW is a ride hailing company, basically a marketplace in the mobility sector, and here I am responsible for the tech hub here in Barcelona that has roughly 130, 140 people dedicated to the supply side. Back in the day, when it was cool to say that we were product nerds or product geeks, that's how I would describe myself. But one, it's no longer cool, and second, the reality is that now I usually say that I am obsessed with driving impact, and I think after this presentation you will kind of understand why.

The six-month MVP

[00:01:22] I want to bring you on this journey, so let's get back to my first day at FREE NOW. After one hour long session hearing all the good stuff about agile, lean, Kanban, scrum, all these good things, and wondering why are they explaining this to me, shouldn't I know all of this stuff, I joined an even more interesting meeting. It was between one of my future teams and some operations guys, and they were asking my team to build a very specific feature that they had in mind. So they were describing the feature, the functionality, how everything should work, what it should look like.

[00:01:58] And instead of my team pushing back or asking more questions, I saw my PM starting writing all the tasks on a post-it, putting them on the wall, and then discussing with the engineering team what was going to go first and next, what was the right sequence. And then all the engineers kind of estimating the effort. And then I saw them all pat each other on the back, super happy with the outcome of the meeting, when they agreed to do a six-month MVP. And yes, you heard me correctly, they agreed to do a six-month MVP.

[00:02:35] Now, I had to confirm that I was hearing it correctly, so I asked them, did you say six-month MVP? And they went on and on about how it was super difficult to build this feature, they had to build this microservice that was going to support this, so it was super complex and it was going to take six months. And then I kept asking, but MVP, is this really an MVP? And they said, yeah, it is an MVP. It sounds weird, but in this company we pay a lot of attention to the quality of what we do, so to release, to ship something with good quality, it's going to take us six months to do everything that operations is asking us to do.

[00:03:15] So I left that meeting half in shock, half disoriented, and I went around the office asking and talking to as many people as I could. I needed to understand what the business looked like, what were the processes, how the whole thing worked. And that's when I got my first diagnosis: these guys were not doing product management, they were essentially doing project management, wrapped around in little nice agile, lean concepts. But what they actually were doing was project management. They used these nice little Gantt-like charts to plan what they were going to do. So essentially they were doing project management.

What FREE NOW looked like 18 months ago

[00:04:02] I want to tell you what FREE NOW looked like 18 months ago. The decisions were made top-down, so either they came from the SMT or the markets would ask us to build certain things. There were inconsistent or non-existing metrics whatsoever, so we didn't know what was going on. Half of the behaviors on and off the platform we could not track, so we had pretty much very, very limited data. Our release cycles were extremely long, so for us to launch something, or run a test, or anything that we wanted to do, would take a really, really long time.

[00:04:38] There was this huge obsession with outputs and not outcomes. So for example, teams would spend six, seven, eight, nine months developing a feature, they would ship it, and then the next day they would forget about it. They couldn't care less how it was performing, not because they didn't care, but because they didn't have data to look into that, and their objective was to ship something, not really improve anything. There was very little experimentation, and I dare to say there was no experimentation whatsoever. So essentially the company was a feature factory. We built features on top of features, and that's how we did it. We just followed the competition and built some things that we thought were essential based off of our own opinions.

Why dual track

[00:05:25] Now, I looked at everything that I had heard, the maturity of the teams, the seniority of the people that we had in the tech hub, and for me dual track was the right answer. I had used dual track before, I had implemented dual track, and in my head, based off of what I had heard, dual track was going to solve a very good chunk of the problems that I was seeing there. Most people have heard about dual track before, so I won't spend too much time on that, but essentially, in the dual track system, teams simultaneously develop the product in iterative cycles while continuously identifying opportunities and evaluating solutions through research, through experimentation, and so on.

[00:06:12] If you want to know more about dual track, I highly recommend that you check all the content produced by Marty Cagan and Jeff Patton. They have online talks, they have a bunch of free articles that they wrote about the topic. They were the ones painting the concept, so please go check them out. If you want to know more about discovery, what discovery is, how to do it, then Teresa Torres is your person. She's absolutely amazing, and she also has a bunch of free online content, talks, articles written, so please go check it out, because it's great, it's amazing.

[00:06:50] So what had to happen at FREE NOW to introduce dual track? I made the mistake of thinking that I could change FREE NOW as easily as I changed my previous startups. What I didn't realize was that a 10 year old company that had over 1,200 people had an ingrained culture, a very specific dynamic, and a lot of anti-patterns that are not changed just like that. It's not as easy as you could do it in a startup that is very prone to changing, pivoting, that is used to this agility in the way that they do business. So I hit a few walls. I'm not going to say that it was easy, and I had to overcome frustration many, many, many times, but eventually I managed to do it, and today I have kind of a very high-level play-by-play to share with you.

Wrestling for control over priorities

[00:07:49] So here it goes. First, wrestling for control over priorities. We need to control our destiny, otherwise we won't be able to make progress. If we continue receiving requests from the executive team, from the markets, there's nothing left for us to do other than plan, execute and ship it. So what I had to do was slowly taking control, or gaining control, over the process, the priorities, how to set the priorities. Now, everybody does this in a different way. In my case we had these approval sessions with our executive team where we were asked to show our roadmap, and kind of how the feature would look like, how long it would take.

[00:08:37] And I used these sessions initially to raise awareness for things that were dismissed or overlooked by the executive team. So for example, I would go there with what they've asked for, I would show the feature, how long it would take, and also the potential roadmap. But on top of that, before introducing the feature, I would say, hey, here is the problem that is leading to building this feature, the problem or opportunity. So I started by framing the problem and letting them know, here are all the reasons why we should build this feature, this is what this feature is going to do for us. And the reasons were always centered around our users. And then I topped this up with data: here's why we should build this feature instead of feature B, C or D.

[00:09:29] Most of the times I didn't even agree with the feature, so it was just the tool to get them to have some visibility over how we should do product. And funnily enough, after the first session, they started asking other departments in the company to come back with the problem statement, all the reasons why we should build this, and also some data. So once this step was accepted, I moved on to a second stage and a second iteration of my process, and I dared to show them an alternative.

[00:09:59] So I would still come with the feature that they had asked for, or that I agreed to show, but then I would use a couple of minutes of my presentation to show them an idea that we had, and how we had come to that conclusion. So I would show them the entire process: here's how we went about it, we looked at data, we did this research, we spoke with drivers, we got this information, then we proceeded to run a bunch of tests, then we got these learnings, we iterated, we eventually got to a high confidence level that would let us build this feature. And this is what we would expect if we were to deploy this feature, to ship this feature: these would be the outcomes, the performance changes, this is how much we would change the needle.

[00:10:47] So slowly I was gaining traction, getting more control, because they were trusting me, they were being more and more open to new ways of doing this, which then led to giving me more control over the process, which gave me more liberty. And with more freedom I would come up with new suggestions, and slowly I started building some rapport and getting more traction, getting more control over our priorities. It was a very slow process, I have to say, but it was essential, because without changing this, without getting control over the process that we run, we're stuck in this cycle of building what somebody else asks us to do.

Finding early adopters and building cross-functional teams

[00:11:33] So the second one, and that happened in parallel, none of this is necessarily sequential, was finding early adopters. We can't do it alone, it's absolutely impossible. This is not a one-man show or one-woman show. You have to find your early adopters, and by early adopters I mean those fearless rebels who either have the same mindset as you, or who buy into your vision and then help you spread the word across the organization. You don't need a bunch of them, but you need some early adopters scattered around the organization who can help you do this influencing work. So they spread the word, they talk to other people, try to influence them to buy into this new vision, and on top of that also bring you back some feedback that will help you preempt the concerns once you go full public. So it's very important that we find these early adopters.

[00:12:27] The third thing that we did was building cross-functional teams. Up until that point the product and engineering teams formed the tech function, and then teams such as UX, so design and research, data, marketing, were seen as enablers. Now, if we wanted to implement dual track the right way, this simply wouldn't work, so we had to move them inside of the teams, and I mean literally move them physically inside of the teams. What this allowed was researchers to become quintessential in the early stages of planning. It allowed data people to be fully embedded in the experimentation process rather than just digesting data for us or doing certain dashboards for us. And then we also brought the drivers, our users, closer to us, by having both engineers and product managers be part of some of our interview sessions, or user sessions, that the research team was carrying out.

Roles, responsibilities and reorganizing the department

[00:13:40] So that was done, and then we moved to changing roles and responsibilities. Our POs became PMs, our teams became squads, and then we changed the responsibilities and accountability. Up until that point we only had chapter leads, so there was no PM counterpart, which means that product had to be responsible and accountable for every single step of the process, both the delivery, the discovery, everything. And this didn't allow the product managers to really focus on what they should focus on, because they were firefighters. They had to plan, they had to execute, they had to be very close to the teams every step of the way, and then they had to respond for everything that was happening. So we created this new function called the engineering manager, or the squad lead, that essentially became the PM counterpart, and they became responsible for the delivery side of the dual track system. So what happened was our PMs recovered time, that now they can invest very heavily in doing proper discovery.

[00:14:46] And then fifth and final, we reorganized the department to promote more autonomy and accountability. Initially we had two big supply tribes. They were split by web development and app development, and we shuffled things around to create four smaller tribes reflecting the user journey. This is flexible, we know that we're going to change it in six, 12, 18 months if need be, but what we needed right now was to create a structure that reflected our user journey. So now we have a first time experience tribe, responsible for hooking our new drivers into our service. We have the driving experience tribe, responsible for the service. We have the happiness tribe, responsible for everything engagement related. And then we have framework, which is more tech driven, responsible for topics such as security, modularization and so on.

[00:15:49] And the reason why we did this was because we wanted to give areas of focus, and also ownership of certain areas, of certain KPIs, more than features, KPIs that they could influence, so that once we introduced OKRs they knew exactly what they could do to help move the needle or achieve a certain key result.

How we work now

[00:16:09] So this was essentially the play-by-play. So how do we work now? These days we have a clear company vision that informs our product strategy. We have implemented OKRs, which allow us to define better tribe level strategies, and then each squad has their own goals, which they achieve by creating educated hypotheses. For us to create this educated hypothesis, we take into consideration a lot of information coming from different sources: users, UX team, the research and design team, data, markets, product, engineering. A lot of sources help us build these educated hypotheses.

[00:16:56] And then we go through the typical process. We prioritize the hypotheses, we do power analysis to see if we can actually run certain tests, then we go through the testing phase using a million different artifacts, and the outcome of this is a certain confidence level. So if we have a confidence level below four we discard that hypothesis, or that potential solution, and we document the learnings. If it's between five and seven we iterate over it. If it's above seven we pass it on to the delivery track and we start executing. So in summary, we moved from a six month MVP to being able to ship something, from hypothesis to launch, in simply four weeks. We're still iterating over this, we want to be way faster than that, but six months to four weeks is kind of okay, so I'm happy with the results.

Three things to look out for

[00:17:52] Now, this process was lengthy, and I would be lying if I said that I wasn't thrown a few curveballs, or that I didn't want to quit many, many times. So I want to share with you three things that I think you should look out for if you are in this process. So first one is, don't underestimate the power of small things, for example sitting people physically together. When we started this process, because of resource constraints that we had, we decided to assign researchers, designers and data analysts to the squads, but they would still sit in a corner in the office, so they would sit separately, and the process was way harder than it should have been. And we realized that if we wanted this to work they could not be sitting physically apart, because they didn't really feel part of the team, the squad. So if you are going through this transition, sit people close together, and if you're working remotely make sure that they're part of the rituals of the teams. Don't underestimate small little things, and especially everything that has to do with human relations.

[00:19:13] Then the second point that I would like to bring out is, watch out for misunderstandings regarding the process. There was a moment where some designers came from a good place, they had good intentions, but they understood that their new responsibility was to take over the ideation part and fill the backlog of our squads in order for them to execute their ideas, the designer ideas. Needless to say that this is wrong. So it's very important that everybody understands what their individual and collective roles are. Make sure that everybody understands what the process looks like, what is expected from each individual, and then what is expected from the collective.

[00:20:06] And then third and final, but probably the most important in my journey, is to make it abundantly clear for the entire organization that dual track is not a linear process. To build good products, sometimes we have to take two steps back, or one step back, in order to take two steps forward. Living dual track is not building checklists and creating processes and documents that squads go through and kind of tick the box and say done, done, done. No, it's like transforming a stale organization into this living organism that gets information from the outside, adjusts internally super rapidly, and then pushes out the right things. This is what dual track is about. It's creating this living organism that adjusts very quickly and builds the right things in the right way.

Was it worth it

[00:21:11] And now, for those of you who are thinking, you heard me say that this was a lengthy process and it took a really long time, it was frustrating at times, I thought I would quit midway through the process, you're probably wondering, was it worth it? And I can tell you it was a hundred percent worth it. Before this journey we shipped features relentlessly, we kept shipping more and more stuff out, but we wouldn't move the needle performance-wise, and I have to say that we were not particularly proud of the service or the product that we had. These days we ship everything with the end goal of changing, improving the user experience, and ultimately what we want is to change performance, and that means also increasing revenue.

[00:22:07] At the same time we are much more efficient with dual track. Just to give you an example, when we started running experimentation in the hub it would take us 22.5 working days and a full team doing the heavy lifting of building the experiment, whereas these days we would do it in one day or less, and probably just with one person helping out. So we became a lot more efficient, we became a lot faster, which means that we can do a lot more experiments, we can try a lot more things, we can ship things faster, and we can change performance much more rapidly. So all in all, everything has changed for the better.

[00:22:50] And now, from a qualitative standpoint, if you entered our office 18 months ago you would see people that got along, they were super friendly and happy and cheerful, but they were very unmotivated work-wise. Now if you go into our office right now you see product managers super motivated, enthusiastic, bringing new ideas to the table, wanting to discuss strategy, wanting to have more and more stuff on the table, experiments, new things to do. So you can see a completely different team in the company. They want to do more, they don't want to leave the company, because we want to change the mobility sector, we want to make our users' lives much, much better. It's no longer about shipping a feature, it's about changing the paradigm of an entire industry, or at least the life of our users, and that's just amazing.

You can start where you are

[00:23:50] And then finally, just for those who are sitting there and thinking, can I do something about this? If you're in my shoes and your organization is still kind of in the old days, and you're wondering, can I by myself do something like this, maybe I don't have the means to do it? There is always something that we can do. So let me just tell you that we were never a satellite office, so we didn't get orders from the HQ, but we are a tech hub, we are apart from the HQ. We are a small team in the grand scheme of things: comparing to the 1,200 people in the organization, we're just 130. So we started in a very small hub, very small team compared to the entire organization, and we had to change, pretty much not change but touch, every department in the organization. Customer care, operations, product, engineering, data, everybody had to somehow change the way that they work, and we did it from Barcelona.

[00:24:53] So now in the company we are seen as the example, we are sharing our knowledge with the rest of our colleagues, and it's been amazing. And I get contacts all the time of companies that want to follow the same path, and I'm just one person. So my message to you is, don't give up. It does change, it does take a long time, it is a very daunting process, it's tiresome at times, and it feels like sometimes things don't change. I went through stretches of two months where nothing seemed to happen, and then all of a sudden things started falling into place. So don't give up, start changing things slowly, and don't get as frustrated as I did, because it doesn't help you much. And just follow this process, because it's beautiful, the outcome is just absolutely amazing, and we're nowhere near the finish line, but we have changed so much.

[00:25:54] And this was our journey. I hope it was useful or somehow inspiring to you, and if you have any questions you can find my email and my LinkedIn here. Feel free to drop me a line. I'll also be available later on to answer any questions in the Q&A. But that's it, thank you so much for watching, and I hope you have a great day.