Aligning Teams Through Product Roadmaps

Marrisse Asuncion at UXDX Community: SE Asia. Video: https://www.youtube.com/watch?v=_h2u8KF7nvQ

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Introduction and agenda

[00:00:00] Hi everyone, I hope you're having a great week. Thank you for signing up for UXDX APAC. Just a quick intro: I'm Marrisse Asuncion, a senior product manager at Quipper Philippines. Quipper is an edtech company that creates solutions for teachers and students, so that we could bring education and make it more accessible to all corners of the world. For today we're going to talk about aligning teams through product roadmaps.

[00:00:33] And just a little bit more information about myself. I actually started my career as a content writer. I wasn't originally from the IT industry, and then I got pulled into project management, and then eventually I just became a product manager and it became official. In terms of the products that I've managed in the past, I've mostly been in the edtech space. There was a small period where I worked for a performance marketing company, but most of my experience is in edtech. In terms of the things I love, I love traveling and camping, so you see a picture of me here with my husband in Tokyo. I also like doing quick morning runs just to get all of the bad juju out at the beginning of my day, and I love playing with our puppy Gnocchi, who is here at the bottom right of the screen.

[00:01:32] So in terms of our agenda for today, we're going to talk about, first, the product roadmap and company vision: how do you make sure that your product roadmap aligns with your company vision as well as the business goals? Next is deciding on the star, so how do you choose a north star that best reflects your company vision, product vision, and how does that all tie together with the product roadmap that you're creating? The next part would be building your roadmap and adapting to change: how do you actually build that roadmap, and how do you make sure that you have room for change, how do you adapt to change if ever you need to pivot? Then the last thing would be setting metrics to measure success. This is where we talk about how you come up with actionable metrics that would help you measure whether your product was a success or not.

Roadmaps, company vision and product vision

[00:02:33] In terms of product roadmaps and company vision: to start, I'm sharing this comic strip of the life of a product manager. I feel like this was me when I was a PM. It felt like I was getting all different requests from all different people. You have the stakeholders, you have the users wanting new features, and also constantly putting out fires, and there are bugs reported. So it was really hard for me to have a solid vision for our product. It felt like I had tunnel vision, like I was just working on the things that are important at the time, but I didn't really understand what the product strategy, what the product vision looks like. So that was really a big issue for me.

[00:03:20] And when the task for me was to create a roadmap for the product, me as a new product manager, I thought that a roadmap is the same as a Gantt chart. So I thought, if I add some features here and there, assign some start and end dates, then we're good, we already have a roadmap. But one thing to remember, the one thing that I've learned, is a roadmap is not really equal to a Gantt chart. Gantt charts are usually used to lay out pieces of work that are supposed to end on a specific deadline. Well, for roadmaps it's supposed to capture your overall strategy, and how you see the big picture, what the big picture for your product is.

[00:04:06] Now just to put that into context in terms of Quipper, in terms of our mission and goals. Our company vision is to bring the best education to all corners of the world. And then in terms of our product mission, it trickles down to the product vision. There are two sides to our product. First is Quipper School, so that's mostly driven by students and teachers, in-class learning or blended learning. And then the next side is the Quipper Video Masterclass, which is more of a self-led tool. For Quipper School the vision is to create a comprehensive and intuitive LMS for flipped classroom learning, and then for Quipper Video Masterclass it's more to create an engaging self-led learning tool, since we want users to achieve their goals through the tools that we're offering.

[00:04:58] In terms of the product goals, there's also a bit of a difference here. For Quipper School their goal is to become the top choice for K-12 and university schools, but for Quipper Video Masterclass our goal right now is more focused on user retention and acquiring more users, since that's the part that we'd really like to hammer on in terms of how the product is working right now.

[00:05:27] And then in terms of the product roadmap, of course trickling down to the roadmap, all of the initiatives for Quipper School are mostly focused on the user friendliness and overall UX of the LMS that we have. Well, for Quipper Video Masterclass it's more, of course, to improve retention and reduce churn, since that's our product goal.

Why vision and strategy matter

[00:05:52] Now the next question that you might ask is, why do we even need to care about vision and strategy? Well, the first thing is it gives your team a clear mission. If you imagine yourself driving a ship, and if you have a clear vision and strategy, then you could sail your team members to whatever destination you're actually going. You all have the same destination in mind.

[00:06:21] Then the next thing is getting buy-in from stakeholders. When you think about vision, strategy, north star metrics, all of that, it's very important to get buy-in from stakeholders, because if they don't agree with what you think should be the vision, the north star metrics and all, then it's definitely going to create friction within the teams.

[00:06:46] Next is building a solid and robust roadmap. If you don't have a vision and strategy, like I mentioned before, I didn't have a clear vision before, so it was really hard for me to come up with a solid roadmap for my team. Then the last thing, of course, is choosing the right metrics for success. If you don't have a vision, like in our case, creating a self-led learning tool to help users achieve their goals, if you don't have that vision then how would you choose the right metrics to make sure that you're actually headed to success? So it's very important that you come up with the correct metrics.

Why a north star is important

[00:07:30] Now we go to the next part: how do you decide on the north star, and what is a north star even? Before we go to that, let's talk about why a north star is important first. The first thing is it serves as your key metric of success. It's a metric that if it moves up then you know that you did something great. The second thing is it guides prioritization decisions. Whenever you talk about which do we prioritize, this feature or that feature, it always helps if you talk about it in the context of your north star, and think about which of those features or initiatives would actually have a bigger impact on the north star. Then that's actually what you need to prioritize.

[00:08:23] Next is guiding your team and stakeholders. I said like a ship, right? So the north star is something that they should be laser focused on. Everything they do should be to improve that north star. And then last is it helps with compelling storytelling. As product managers we're all expected to be great storytellers. We create all these presentations company-wide, even with stakeholders. So if the north star is clear, then it also helps you sell that story: look, we released this, and this very important metric actually came up, so something good is happening.

[00:09:05] So in Quipper's case, by the way, we want our users to achieve their learning goals. How do we think they can do that? We feel that they'd be able to do that if they watch videos, complete courses, or book our tutoring and coaching services. So our north star could be percent monthly paid users who watched videos, completed courses, or booked tutoring or coaching services. It's a mouthful, but since we have a lot of core product offerings, then we could have a different north star for those different offerings. And if you think of it, I said percent monthly paid users: it tries to cover the conversion part, which is the paid users, and also the engagement part, the part where they watch videos, complete courses, etc.

[00:10:02] Now in terms of some examples, you see these big names here, Spotify, Facebook, Amazon. They actually have north stars as well. In Spotify's case it's time spent by subscribers listening to music, since they're subscription-based, so that's very important, but they also try to measure engagement, make sure that they're engaging users in terms of the time they spend in their platforms. For Facebook, since they serve ads, monthly active users is very important to them. And then for Amazon, number of purchases per month is of course very important since they're an e-commerce company. And it's not like a metric like number of items added to cart. Those could be input metrics, additional metrics that they could track, but the ultimate metric is really how many purchases they get, because that's what leads to conversions.

Deciding on a north star: the engagement game

[00:11:06] Now in terms of deciding on a north star, there are four steps that could help. First is you have to understand your engagement game, so that could be attention, productivity and transaction. I'll talk about that more in a bit. The next thing is about choosing actionable input metrics. I said that if a metric is not actionable then it might not help a lot, because it doesn't tell you what your next action should be if ever that metric goes down or up. Next is you have to understand your critical events and success funnels. I'll be talking more about critical events and success funnels in a bit. And then also conducting workshops with your team and also your stakeholders.

[00:11:56] So first thing, understanding your engagement game. There are actually three types of engagement game: attention game, transaction game and productivity game. In terms of how they work: for attention game, the main goal is to maximize the time the users spend in the product. So perfect examples would be social media sites or sites that get revenue from ads, so Facebook and YouTube. And then for transaction game, the focus is more on helping customers make purchases with confidence, so this is common in e-commerce sites like Amazon and Alibaba. And then for productivity game it's more on creating an easy way to either complete tasks or help users gain mastery over a task, so it's common in B2B software, like for example Jira, where their main goal is to make sure that teams can easily track their work and complete the tasks.

[00:12:56] But in Quipper's case, we also sit in the productivity game, because we want our users to gain mastery over something. So some product managers could say, but all of these engagement games are actually important to me. And it can be true sometimes, but it's important that you choose one, so that that's the main thing that you're going to focus on in terms of the north star metric that you're going to pick.

[00:13:25] Here is a graph on customer value exchange. This will also help you understand where you are in terms of the engagement games. For example, attention: what they offer to users — like Facebook, let's think about Facebook. Facebook offers to users entertainment, because they can watch videos or read through posts from their friends, as well as information, then also self-expression, so they could express whatever on their pages. Transaction, it's more about utility, status and fulfillment. There's a need for a user and they need to fulfill that need by placing an order. For productivity, there's also a bit of utility there, but the main thing is efficiency and mastery, so as I mentioned, we want users to be efficient at something or master something.

[00:14:20] Then what the customer gives back to you in return, for attention these things could be ad engagement, subscription, etc. For transaction, the purchases, if they purchase your products. And then for productivity, subscriptions or upgrades, if you're subscription-based.

Choosing your input metrics

[00:14:42] Now in terms of choosing your input metrics, it helps to visualize it this way. Try to break down your north star metric into four different dimensions. So you have your north star metric at the top, and then you have breadth, depth, frequency and efficiency, which are all meant to support your north star metric. For the breadth, that answers the question of what the reach of your product is, how many users are you actually reaching. For depth, it's more about how much users are engaging with your product, how deep do they go in terms of their engagement with a product. For frequency it's more about how often they engage with your product, do they come back every day, do they come back every week, etc. And then for efficiency it's how efficiently you're delivering your product.

[00:15:38] So let's look at an example here. These are some metrics that I've used in the past for a fair[?] product that serves ads. The goal of the product was to help students find their best university match, but the way we were generating revenue was from ads. So our game was definitely attention game, since we were driving revenue from ads, and our north star metric was time spent on site. So we'd like users to spend as much time on the site and apps as possible, so that we could generate more ad revenue.

[00:16:16] In terms of the input metrics: for breadth, registration rate and number of site visits, those were very important to us, because that also helps with the amount that we could potentially get from ad revenue if users register. And then for depth, the time they spent per session as well as the number of meaningful events per session. The meaningful events per session is supposed to augment the time spent per user per session, because we want to make sure that they're performing meaningful events, like for example saving schools, curating their career list, etc. Because we know that if they're performing those meaningful events then they're engaged and they're going to keep coming back to use our product.

[00:17:05] For frequency we used the number of sessions per user, so how often or how many pages they consume when they're on the site. Then retention rate was also important. For efficiency, bounce rate was one big thing. How do we know that we're delivering the information that they need? That is, if we have a relatively low bounce rate. And then app crashes, of course: if the app keeps crashing then we're not at all efficient in terms of delivering our product.

Critical events and success funnels

[00:17:40] Now we're done with the input metrics, we're going to talk about critical events and success funnels. A critical event is an action that aligns closely with your core value proposition. So for example in Quipper's case, our core value proposition is, if users use our site or our apps, watch videos, book tutoring or coaching services, it's going to help them achieve their learning goals. And you can also think of it as the action that you want to drive your users toward: what's the main thing that you want them to do as soon as they visit your site or open your apps?

[00:18:26] Now in terms of the number, can I have multiple critical events? It's okay, especially in our case, like I said, that we have distinct user flows, so we could have different critical events per offering. But it's good if you try and limit it to one critical event per core product offering, just so it's not all over the place.

[00:18:52] Okay, so some examples here. For Airbnb, the main user segment that they're catering to is someone looking for accommodation, so of course the critical event would be to book lodging. For Instacart, they're catering to users who need groceries, so the critical event they're tracking is scheduling a delivery. And then in Quipper's case we have different user segments. There's actually more, but I just focus on these three. First is the student, so of course we'd like them to complete courses or assessments or quizzes, etc. For teachers, we'd like them to assign activities to classes. And then for learners, the self-paced learners, we want them to complete recommended content, videos or courses, or book tutoring or coaching services.

[00:19:46] Now in terms of critical events and success funnels: success funnels, think of it as something that helps you understand how your users progress through the different steps that lead to your critical event. So for example, here you have the biggest part of the funnel, which is the registration part, and then we have the critical event here, which is completing a purchase. If you know your success funnel, if you understand your critical events, then it would help you understand where your users are dropping off, and then you could also focus on optimizing that part. So for example, if it looks like they're dropping off in terms of browsing content, then you might want to think about ways to surface more relevant content, be it through recommendations or through better search results.

Workshops with your team and stakeholders

[00:20:49] Now the last thing is you also have to make sure that you conduct workshops with your team and stakeholders. As I said, the north star framework will be successful if you get buy-in from them. What you see on the right here is a design thinking exercise called a creative matrix. And if you wanted to decide on breadth, frequency, etc, those input metrics, what you could do is you could keep the rows at the top as your segments, and then you could have breadth, efficiency, etc, here, the four drivers here. And then the sky is the limit: when you're in the workshop, encourage everyone to post all of their ideas on the board. And then when you're conducting those workshops it might also help if you have a RACI chart — responsible, accountable, consulted, informed — just so you know who you actually need to involve in those workshops.

Building your roadmap and adapting to change

[00:21:52] The next part is, how do you build your roadmap and adapt to change? One thing, when I first heard of the north star framework, I thought, is it something permanent? I realized that technically it shouldn't be permanent. If the product strategy or company strategy changes, then you're going to need to be able to adapt your north star metric to such changes. So for example in Quipper's case, we're currently focused on retention, but what if we focus on something else? Our team will need to come up with a north star metric that matches that. Just remember that one quality of a good north star metric is it needs to be relevant for years, just like a product vision. It shouldn't be short term, and it's something that should guide you through the years.

[00:22:52] Now, in the agile manifesto, since we're talking about change, one of the key things there is responding to change is more important than following a plan. So when you build your roadmaps, especially if you're in an agile organization, you really, really need to make sure that you could respond to change if ever.

[00:23:16] Here's another comic strip that I like. When a product manager prioritizes, they could score each feature, list the features, use the RICE method and all, but then at the end they end up building whatever upper management likes, which I think is still pretty common.

Top-down versus team-driven roadmaps

[00:23:37] Now I wanted to talk about top-down and team-driven roadmaps. For top-down roadmaps, that's mostly the comic strip that I showed earlier. So the CEO or founder usually sets the agenda for the roadmap. He also spells out how it would be executed. This is very common in startups, especially if the founders act as the product manager since there's no product team in place. For team-driven roadmaps though, it's kind of different, since the product manager needs to collaborate with the stakeholders as well as get the user feedback and synthesize all of these sources, so that he or she could come up with a solid roadmap. They're also going to need to ideate the possible solutions with the team, so it's not something that they're mandating. And then they craft the roadmap and get buy-in from stakeholders first before they actually lock it in.

[00:24:40] Now, what do we think is better, top-down or team-driven roadmaps? When I was doing a comparison, I still feel like team-driven roadmaps are better. The first reason is, when you do top-down roadmaps, the process of choosing the priorities is not democratized, which is kind of hard because it could create friction. Well, in team-driven roadmaps the entire team collaborates, they all contribute to the roadmap, so that naturally leads to alignment.

[00:25:16] Now in terms of the product team, in top-down roadmaps the product team also becomes order takers. Well, in the team-driven roadmap there's more autonomy and they decide on what would bring the most value to the product given the information that they have. The last is, in a top-down roadmap approach it starts with the assumption that there's this one person who knows what's best for the product. Well, if you're in a team-driven approach then you could synthesize ideas from all sources and take those into account as you build your roadmap.

Tips for building a roadmap

[00:25:56] Now, some tips on building a roadmap. First is just make the most of whatever roadmap approach you're taking. So whether you're doing top-down or team-driven, that's fine, and if you'd like to switch to team-driven then make sure that you get your buy-in from stakeholders first. Next is to understand your user segments or personas. You won't really know what to offer them if you don't understand their pain points and their needs, so this is very important. Next is to conduct a lot of user research, like lots of them. So do FGDs, do user interviews, just so you actually understand what your users need.

[00:26:39] Next thing, conduct regular design thinking sessions. This could be with your product team or stakeholders or combined, just to make sure that you're all aligned on the initiatives that you'd like to pursue. Next thing, build everything in small increments. We have to adapt to change, so we always have to leave room for change. Instead of just going all out, releasing the perfect feature that you think is going to be best, it's better if you release a small part of it, test it, see how users respond to it, and then iterate after.

[00:27:14] The last thing is of course, beware of the feature factory trap. It's always good if your team is accountable to outcomes, specific outcomes. So instead of talking about features, try talking about outcomes instead. This is very challenging, like in my case as well, it's really hard to avoid this feature factory trap.

[00:27:40] This is just an example of how you could try and prioritize your roadmap. This is an impact-difficulty matrix. This is one design thinking exercise that you could do with your stakeholders, and it lets you visualize which are the ones that are quick wins, which are luxuries, which are the ones that you should probably set aside. It's good if you do it in an actual workshop and not just through a document, because if you do it through a document, everyone goes off and updates the document themselves, you lose the conversation part, you lose the discussion part. It's like you're just looking at numbers. It's better to just have a workshop with them instead.

Setting metrics to measure success

[00:28:24] Okay, so we're in the last part of my presentation: setting metrics to measure success. When you're choosing the right metrics, I think there are three things that you need to remember. First is your metric should reflect your product vision and your goals. As I've repeatedly said, whatever metrics you choose, it should be aligned with your product mission as well as your goals, which of course naturally makes it align with your business goals and company vision. You are also going to need to conduct a lot of workshops with your team. So metrics is not something that you just think about and then you just cascade it to the team. It's better if you get them involved in that process. Then of course they should be measurable and actionable. If it's not actionable then why bother monitoring it?

[00:29:18] Quick reminder as well, to avoid vanity metrics. Vanity metrics look good on paper, but the thing is they're not really a good reflection of the success of your product, and they're not actionable. So for example, trial users, that's less relevant than converting users, because here you actually measure the number of users who are converting, if that's the end goal of your product. I'm not saying that vanity metrics are something you should ignore. Yes, you could still look at them, but they shouldn't be the main focus of your energy. You should be pouring all of your energy on the actionable metrics.

[00:30:04] In terms of tracking your progress, it's also important that you build and maintain a dashboard that's visible across teams. That's also how you can communicate across the teams about the status of your product, whether your product bets are winning or not. Then of course, always try and start conversations with a review of your metrics. So if you're having team meetings, it might be good if you look at your dashboard and then talk about, okay, we released this, and these metrics went up, these metrics went down. So it's going to help you frame that conversation.

[00:30:46] Related to that, you also need to understand how the things you release impact your input metrics, so you know what your next step would be. And then of course, just remember that when you say that something is released, it doesn't necessarily mean that your product is successful. If it does not move your key metrics, your north star metric, your input metrics, then it's not necessarily a success. That's hard to hear. Yes, you could celebrate after a big release, but still, at the end it's the metrics that you should look at.

Takeaways

[00:31:23] Now, what do I want you to take away from this talk? The first thing is that having a clear product vision is the key. You won't be able to build a solid roadmap without this. The next thing is a north star metric with supporting input metrics will help you build a strong and solid roadmap. Without a north star metric it's going to be hard for you to come up with initiatives that would actually support the success of your product.

[00:31:55] The next thing is driving team alignment is at the heart of building a great roadmap. So if you involve your team, if you involve your stakeholders when you try and build your roadmap, then it's going to naturally lead to alignment. Next, great roadmaps take a lot of work. As I mentioned, it's more than just an Excel sheet, a Gantt chart with release dates. It's really a combination of all of the different things that you think would contribute to your big picture vision.

[00:32:31] Then of course, lastly, a roadmap shouldn't be set in stone, so always leave room for changes. When you say lock it in, it's not really locked. Yes, you lock it in maybe for the sprint, but for the succeeding items in your roadmap, just make sure that you could also pivot if needed. Say you released a feature and it wasn't received well by your users, then you might need to pivot and come up with a different approach.

[00:33:04] All right, that's actually the end of my presentation. I just wanted to share that we also have some openings. You can scan the QR here if you're interested. So come join us if you'd like to become a distributor of knowledge. All right, thank you. Thank you for attending.