Planting The Right Discovery Environment

Hey ladies and gentlemen and welcome to ‘Planting The Right Discovery Environment.’ This talk explores the ecosystem that the product team in sofatutor developed for enabling discovery work and I will share our learnings from the change and why it is important to not only have the right tools but also the common ground on which the team can bloom and bring sustainable success.

I will talk about the team structure, the product work and the role of management and this is not only directed to product managers or designers or developers but also to the managers and people who want to make real change at their companies also for example known as entrepreneurs.

But let me shortly introduce myself, my name is Soeren Von Zebelin and I started my career as a producer at Goodgame Studios which is now known for titles like Goodgame Empire and Big Farm. And there I grew with the company as we scaled from roughly 100 to more than 1, 300 people in just 4 years and I was responsible for the success of scaling teams, product teams basically of around 7 to up to 60 cross-functional team members. And within that work, I loved the challenge of creating the most efficient process possible and while keeping the goal and certain conditions in mind obviously but slowly the question of purpose developed in my mind. And I wanted to do good and bring value to the people with my work and that’s why I started falling in love with the real product work. Always having that value in mind that you bring to the people, so product nerd was born - yay. Since then I can’t stop thinking like a product manager even at work or on vacation and all the books I read, all the talks I enjoy are about product and their value. Maybe the only thing that gets me off of my product mind are boardgames and surfing, luckily, because otherwise I’d probably drive my wife crazy. And this is me standing in Galicia doing a workation for about 2 weeks and for my wife and me it was the best time this year and I know there wasn’t really big competition but still only product work and surfing - it was freaking amazing!

So the wish for bringing value to the people brought me to education, since I personally experienced the flaws of the system. I still think like it seems to be one of the least developed systems in Germany next to the internet reception, so after working at one and one _*wink wink* _I moved to wonderful Berlin to work at Sofatutor as a product owner.

Sofatutor is an e-learning platform providing German pupils, parents and teachers with thousands of learning videos and exercises and we cover nearly all the content of the German School Curriculum. Here you can see a little map from our exercises. Sofatutor former Head of Product from 2 years ago was a person I knew from before. Luckily so even before I started I knew that a seed of product discovery would find a nutritious soul to grow in. And after I recommended the book, Inspired from Marty Cagan, it seemed to have had the impact that I hoped for because after about a month he sent a colleague and me to London to a workshop with Marty Cagan. There we were able to learn more about the post-agile product management - that’s how we called it back in the days, it’s actually not so long ago but it feels so long ago because so much has happened. Our goal was to do a top-notch product work and to learn from the best and bring this to Sofatutor.

I have to say that all the experiences and changes in the company that followed, clearly were not all driven or initiated by only me but rather by an amazing team effort from people with the genuine wish to make things better and I have the honor right now to walk you through some major learnings of this change, to grow a sustainable environment for product discover. As I already said, but spoiler alert, we’re still far away from the optimum and we’re also just learning but I think this is already - we’re on a stage where we can talk about our learnings.

So let’s first talk about the team structure and how to water your plants. As I started at Sofatutor, the product team and the tech team together was already set up in agile development teams. So the transition was about 1 year in that past and they were already for comfortable with sprints and iterative work. They’ve tried to keep major topics in each team and mostly based on the knowledge and past projects from the developers and the PO’s. Kind of the paths from the teams were still orbiting all around something, we’ve called the responsibility map. It looked something like this. Here you can see each can every part of the platform had to be placed somewhere unto this map, so this is a classic example of component based teams and it caused a ton of discussions and attempts to somehow get it right. And the result was that the team felt like servants, mercenaries responsible to continuously work on X, Y, Z and not testing if it actually made sense of a difference on the platform what they’re working on. And next to this all possible stakeholders were equally mapped to the teams and what happened is that we tried to water every plant on our list, just like this picture. That was the background by the way and you can imagine that we didn’t have enough water and resources to really grow any fruits in this way. We just kind of tried to keep the most important plants alive, more or less. In our talks about product discovery, we decided that we want missionaries and not just mercenaries. Just as John Doerr says preachers and people should want to make a difference with their work and feel success and failure and harvest the fruits of their work.

So one of the major changes led by our Head of Product and the management teams was to establish team missions. Those were closely aligned to the team knowledge and personal preference from the team. And so it was really actionable. That was good because than we could like really deal with that already and it was super important. Additionally to that, to get the legitimisation from the management to act and focus on those missions that’s why it was good that they gave them. Like they talked with us about it and they gave them to us and it was like we felt empowered to do, to work on this and over the course of the following months we gradually decided for and against components and KPIs that helps us with our missions. We were allowed to actually make bold moves as long as we could explain reasoning to all the stakeholders, so we learned to focus on what matters and we communicated it properly. What happened to the responsibility map actually - like i just introduced it to you - the whole responsibility map basically turned into a contact list for stakeholders and nothing more so they basically when they have a request or whatever in this and that feature on the platform and the thing is that this led to the end of those endless discussion. The thought is like, okay we don’t have to get it right. We basically just make it the missions and get better at this, instead of somehow discussing the abstraction level of the items on the responsibility map.

So the mission of one of our team was all about elementary school kids and we focused our efforts extremely well on understanding the children afterwards and how they learn and what is fun to them actually and how we can bring such an experience unto our platform. This was one success story resulting in this overhead that we also showed in the end, and this success means now that we have a super fun way to do exercises for elementary school kids all led led by Pommes the penguin - Pommes is a word play in German it’s like fries - but it turned out that it’s a very, very famous name for pets. So our product manager and that team made a very cool move to just go with that and they actually love it, kids love it! So that’s like the first result that we had after changing it to the missions. Another example is how we managed to reimagine the whole navigation of the platform and reduce the path to get to the content by three big and hard steps. This is the first screen of those steps, so we crunched it down to the most necessary parts.

So our main takeaways were the following: transpose the component driven to mission focused teams, learn as a team what to act on and what not to act on and communicate those moves and decisions. Be bold and outgoing as a team and show the results.

The next area I want to talk with you about is product work. I always understood product discovery as a major change in how a product manager works and thinks. And one of the biggest problems occurs when product people don’t know what assumptions their work is actually based on. The moment you start realising that basically you don’t know anything. You realise that you to learn, to see - maybe for the first time in your career. A few year back, we had very little knowledge on the actual behaviours of user, their intentions, wishes and everything we saw came from isolated analytic platforms. Our database which was for our type of product was very superficial and once we understood how little we know, it felt like walking in the dark. Just like this here and he had a 6 so that’s in German school system, that’s a very bad thing - but he still smiles so whatever. I mean probably because he didn’t know better. We know, okay, we’re walking in the dark here and we also know that discovery and plants need sunlight to grow good. I know plants grow also without sunlight but they have no colour, they have no shape, the do have shape but they’re missing something very important. So what we did… First we tried to find the best ways to do systematic generative and evaluative research. I mean systematic, so we can go over it again. I call this Learning First. We need to find the problems to solace and then understand if our approaches to solve that are working. The basic product discovery one-on-one, I would essay so we integrated Amplitude, Hotjar, Survey Monkey and made tons fo user tests, interviews, surveys and learned best practices on how to run A/B Tests. Here are a few examples of thing, tools that we basically use at sofatutor and also we use Slack. In there, we set up an automation that forwards any type of user feedback from the platform directly into the Slack channel to the company and now whenever we do something, we learn quickly as possible directly from the user where something went wrong.

Let me run you through this with a very straightforward example. The quick start with Hotjar service, we managed to get our hypothesis confirmed that around 35 - 40% of our users want to start where they left off in terms of navigation. This is quite a big deal, that means people get to the platform and they actually just want to start where they left off. So when we give them this possibility, 40% of our users directly get to the content super quick. That’s what we want to grasp. That’s the value we want to bring. So after gathering a lot of ideas on how to solve this we ran through some usability tests via look back with our users recruited by testing. One solution seemed to work and with this, we recently deployed and A/b Test supported by convert now with Amplitude. We realise that the metric is not getting better but according to the A/B test guide even significantly worse since we know there is a big potential.

We will have to iterate over it again but just imagine what it would mean if we would just take out 1 or 2 of those steps, we probably wouldn’t have acted on it at all. We wouldn’t know that there is this potential, we would probably just leave it on the platform even though we know that it makes things worse. So this feedback loop that I’m talking about, we need to walk the sun, we need to know what to do, we need to know where to go and what to do so never before we were able to make as quick and well informed decisions and we never felt closer to our customers and users. Everything is measurable, all the doubts of naysayers could be debunked and we stopped holding us back from actually really making a change and bringing innovation to the table.

Something we need to take care of right now is to make sure everyone who should now about those insights is really up to date and can understand the reasons behind decisions and also to be able to take the right decisions. So working with reports and presentations, workshops and documentation is actually key. For this we need to take extra time and invest as much love into this as we put into gathering the data and forming insights. A few tools that seem to work for us are the opportunity tree. That’s just an example of this, I didn’t fill it in but I think you get the gist. We also built user story maps with the team, when we make user test we also have it to be hypothesis driven. We tried customer letters which worked well and also have dedicated weekly meetings and especially the latter now seems to be the core of a functioning discovery work and alignment. It helped the team to keep talking about the why rather than about the what or how because that what in the home or less actually has dedicated spots, sprints and scrums. Just talking about the why seems to never really have the spots so we just got in there and we called it Discover Lab.

The main takeaways are Discover Learning first, create valuable feedback loops, make your learnings transparent and actively talk with your team about the ‘why’.

And now let’s get to the management, as we came back from the workshops we started our tour through the company, we gave tons of presentations to people telling them what product discovery is and what it means and how we imagine it to be at Sofatutor and what our work would cover when we got the water running. We’ve talked to the development team, to the product team and the design team, to the management to whatever stakeholder you can come up with - we wanted to make sure that everyone in the company understands this mind shift.

In the beginning, it seemed to be very successful and people gave the thumbs up and they were like yeah sounds cool, go for it and whatever you know. You can see two presentation slides like the first one is the original from when we gave the presentation of what actually product discovery and the second is what we called the core team. Like what core functionalities do you need to cover in the teams and also like this, we try to make sure that people know what is going on. Unfortunately, we’ve realised that there was a big gap growing between management and the product teams and it was all like lending to a lot of miscommunication and issues in the strategy process. If we didn’t do anything things might have turned, messed up big time. First we needed to understand what cause this though, so we came to the following conclusion that product discovery. Yes, I know, seems like something we could know before but knowing and realising are 2 different things. So clearly we heard that from Marty Cagan and from all the books that I read or also my colleagues read that we could have known that so we though it's actually necessary or it’s only something that is drastically changing something at the work for product teams. After our tour, no one was aware what it actually means effectively and practically for their work and how it was supposed to change so that the discovery can actually bring sustainable success and most of all the management didn't know about their participation. Hard work is absolutely necessary, they provide the soil for the seeds to grow in, so yes, you need to get them on board and this is something that we had right so we gave all those presentations. We had them on board so what did actually mis, we should have planned clear action items for them and for us. We should have talked through this and I don’t want you to make the same mistakes and that’s why I am trying to be as clear as possible and basically what those action items are supposed to look like.

So we’re in a process now at Sofatutor and bear with me, it's nothing that gets done within a month or two, it's a process and you also need to acknowledge the process. Number 1: you need a product vision. I know you've heard it already, a lot most of you probably when you heard about product discovery and stuff this is important or you need it or what is it and stuff and I'm gonna make damn clear right now you need it. If you don't have it in the long run you're gonna run in the wrong direction so you need to realize how important a sustainable product vision and the discussion about it is. This builds the actual foundation of any discovery work, see it as the soil to grow your environment in if there isn't the right composition of nutrients, hardly anything will grow or the wrong plants and sure it's super hard to create a good and compelling product vision, i know that but it's absolutely necessary. Design a clear reporting process. That's something that I learned the hard way hard way, how and when do you share insights, who challenges, what type of insights are to be shared and how are you as a company learning together when a flower dies or a plant dies like a bad performing experiment. It gives important nutrients to the ground and for next plants to grow on right so they like dissolve and whatever and then the ground has a lot of nutrients and then on this next plants can grow make sure you're building on top of that it sounds weird but honestly don't scare away from the idea of writing a whole paper of learnings and insights - to share insights that’s a lot of work will be based on it's freaking worth it. Provide a clear strategy intent or business goal and this needs to be absolutely transparent. Management and the teams or at least the product managers need to work through them together. Is it adding customer value or capturing value also known as innovation and optimization? Is it cycle time? Is it ideation or growth? Fitting to your vision obviously and if there is no such thing as a clear strategic intent in best case the product discovery brings success that you don't care about and in worst case it simply goes nowhere. Last but not least set up a fitting type of metric. I'm saying fitting because you need to find out what's the right way and you need to do that collaboratively and transparently if you don't have this the team gets confused at some point at least if for example retention or activation or adoption or task success are the keys to success for the company right now and figure out if it's private metrics, if it's the hard framework from Google or the nostal metric from Amplitude it depends a lot on your business and on your organization but make sure you find the right one because you need to work this out together.

If you remember the sunlight in the forest there's only so much space where it shines right and just like on the market you need to make sure you can grow at exactly those spots so.

Let's shortly go over what we've learned today. Probably was a lot and I want you to go home today with three major learnings: number 1, first how to water your plants. Transpose component driven to mission focused teams, learn as a team what to act on and what not to act on, communicate your moves and decisions and be bold and outgoing as a team and show the results.

Number 2: let the sun shine. Discover learning first, create valuable feedback loops, make your learning transparent and actively talk about the why with your team.

Number 3: Provide nutritious soil. Formulate and discuss a product vision, design a reporting process, provide a 100% transparent and clear business goal or strategic intent, work out metrics or OKRs in the teams.

I hope you all will get it right too and make the best out of your own mistakes and then learn together to grow on top of that so that your teams can actually make the change that your company and your users are longing for. So Thank you for your time and I hope to see you later today at 6:00 pm Berlin time for the Q&A.