Continuous Learning: Feeding UX With Monitoring

Thank you. Okay, I'm going to do this by hand because I'm using my own thing. So, this won't work. Anyway, we are here today. And I want to tell you a little bit about how we work at my company Next games. And this is all about monitoring. So basically, data and how we feed it into, how do these become features, I'm going to walk you a little bit through a player segmentation process and what data we have been gathering and how with this kind of insights that we gain, how do we develop features, you know, what is our process there. So, on my agenda, I will tell you a little bit about myself and the company that I will be working for. And I will also give a quick introduction to games as a service, because that's our business model. And I think it's really important to understand, you know, what KPIs are actually important for us, in this kind of model. So, I will give you a quick introduction, then the beef up with the monitoring and the feature development.
So, let's get started because we're already behind, right? Okay, so a little bit about me, I've been a UX designer for a couple of years, I have started my career in 2006 as a graphic designer for a really long time, I did a lot of editorial actually, that's where my love for information hierarchy comes from, even books and then later digital products where, readability likability, understanding information that is for me, at the core here, and the company that I work for is called Next games, it's a mobile games company. We don't necessarily come up with our own works, we actually work with franchises.
So that means, we join franchise holders, so people that have big IPs, and create cinema movies, series or books. And together with these kinds of companies that create these fantastic products, we create the games for these kinds of IPs to bring to the fans to enjoy these kinds of games based on some IPs that we work with. The biggest one is The Walking Dead, we have actually two Walking Dead games, the latest that came out as a location-based game, where you can play like Pokémon Go, but instead of, gathering cute monsters, you actually shoot zombies with your friends. So, it's a bit different. But our first Walking Dead game was a very classic RPG strategy game. And that is the one that I'm going to talk about, I worked really long on that. And we gained a lot of insights. And that's what I'm going to share with you. I want to give you a quick 32 second trailer for the game, so you get an idea, you know what you actually need to do.
So, let's take a look. Actually, there's some kind of sound okay, but if we don't have any sound, that's fine, too. So basically, you shoot zombies, you slash zombies, just like in The Walking Dead series. You know, you collect these awesome heroes that you have, you know, just saw yesterday in the TV show, you know, Neguin with a super cool weapon, you can collect everything around that you can level them up, typical story. Role Playing Game, what is specific about the genre of this game is that it is a turn-based, that means you have a lot of time to think. So, this game is not super active, you need to do things really quickly. But instead, you have time to think and it's very strategic, basically the elements of the game and how to advance that is what this game is all about.

Okay, let's go to the next slide. So games as a service, you know, mobile games, you know, this new kind of, well, newest kind of revenue model that you know, nowadays is really popular is the free to play actually, where this all comes from is called this kind of umbrella term games as a service and that exists you know, not in the last decade but this started in the in the 90s with all the multi massively multiplayer online role playing game so this is nothing new. It's just something that has kind of evolved with like new models inside Free to Play is just one model inside games as a service, which games as a service basically means or what has been created for us to gain revenue still after an initial sale of a product.

So, you know, back in the days you had, also nowadays, there's two games you pay $60 And that is it. Even though nowadays are a lot of updates coming you can pay for usually a classic model of monetisation When was this initial, say you had a really long development pipeline, you paid for the game, and you know that it was kind of at then usually the development teams moved on to, you know, to the next thing that will take many years, this has changed significantly. So, what came with this is these, you know, really massive, you know, worlds that we are actually crafting that have a really big sense of community. So that that is really what is in there at the core, you know, all of these free to play kind of games, or really many of them are actually still have this MMO, this massively multiplayer player online gaming thing at the core, they are all life, they are all online games, that get continuously developed. So, this is what this business thing is all about to have this kind of product that I think many of you nowadays, you know, I think in all the apps, you know, which we try to not have these kinds of things that we make and forget about, but we learn about them now with all this data that we have and try to improve them.
So, and that is the same way that this kind of works. And the important thing about this is the free to play thing, because with free to play, there are, you know, very specific KPIs that are important for us. So, these are the three really main ones. So due to our games, being free to play, you know, players can freely download them and play with them, there is no initial sale, we haven't converted anyone, we don't, we haven't gained any monetary value in that sense. So, we need to retain these players, which means that we need them to come back to our game, we need to engage them with our content. And then in the end, if we did things right, and people actually had fun, and they had a good time, we can convert them into paying users. And then this is where all of this kicks in, then that's where the business basically lies. And that is really important. Because the monitoring that I will show you the data that we collect is all about how do we improve this, you know, how can we get our business to thrive? Basically, it is these three KPIs key performance indicators that we really care about.
And the engagement really basically stands for fun and at the same time, it's for retention, you come back to a product, because you know, you had a good experience, you know, that is all, you know, what we are trying to do, basically create a good product, just like any one of you, we also have a little bit of how would I say this, you know, many of you work on apps that actually solve problems, you know, maybe with your app, I can call a cab or you it takes care of my banking, you solve problems. So there's a bit more value, or maybe even to some extent dependent, you make my life easier, but with the products that we create, they are pure entertainment.
So, it's already a little bit, easier for people to drop these. Someone can download our game, and then there were technical problems in the start, or they didn't understand something, or it wasn't as engaging, it's very easy to drop. So, for us, it's even harder, this whole retention thing to get people to come back, you know, no one else is, you know, relies on having these apps, you know, that is good to know, and something about the market. Also, this whole retention thing is important. It's an extremely expensive to do user acquisition, I think the current dollar value per user is around $1.
Due to the nature of games, a lot of people churn or they kind try the game, and then it's not actually what they were looking for, I'm not as interested. So just imagine this huge amount of users that you paid for coming to the game who actually don't stay. So, that is not a good business case. So that's why we work on these kinds of things. So yeah, let's move forward to the monitoring. We have the same daily metrics, daily active users, how many new users come in, what's the average revenue, and then of course, the retention, which is really important for us and due to the nature of games, a player who logs in at day one has a very different experience than a player at day seven.
So, this metric is really important to us. Because the player they want you to play the game's nature has a different experience because the game changes you know, you progress through our product, in the start at a very fast pace. If you unlock new features, the whole product actually can look visually very different to you, depending at what point of progression you are. So, there are a lot of very specific learnings and very specific parts of the game and the flow and the experience that we actually need to care about and look at. But the daily metrics only goes to find maybe on the day to day basis, hey, you rolled out a new update, and you find, okay, something changed in the retention, maybe there are technical issues, that box you can find stuff, but how do you really improve? You know, how do you really improve these basic KPIs to retain players and to engage them and maybe to convert them? For that, you need to actually learn about your audience, you need to find the segments, who are these people that are interacting, you know, actually, with your product, and that's where we needed to kind of find a way to find out about them.

So, we came up with this initial set of questions, you know, like, how could we go on about identifying these kinds of audience segments, we call them also player types. So, we asked questions, and as you saw earlier, maybe in the video, there are a lot of ways to fight against zombies. And we have a lot of features that do that. And they are all missions. So different mission types, these are our main features where there's actually some kind of actions, you know, you interact with, with the gaming world, you do your strategy that it's called, also sec to sec, it's a second to second kind of gameplay, you know, where you where you really like to where you have maybe most of the fun also, let's, let's say like that. So anyway, it was, we have a lot of these features that do kind of similar things, but in different ways.

So, we were like, Alright, so what missions do players play? What is their favourite way of slashing these zombies? Because there are so many, maybe by looking at them we can find out. And from this kind of questions, more questions came up, for example, how do these players progress than related to this player type, because there is a lot, you know, a lot, lots of things going on in there. Then there is also of course, in the long run due to again, the nature of the game that changes drastically throughout the lifetime of these kinds of players due to different things happening and unlocking. So how do they transition maybe between these player types, you know, an early segment, you know, of players, you know, looks very different and has a very different experience.
So, when they move on and get more experienced, you know, how do these kinds of players look, what are they doing? And then of course, we also have business related questions like, how are these players spending their time. So, we did this approach, we took one week of data, the main measure was missions played per player like and what kind of missions it's kind of a snapshot, you know, of the overall of the game at that time. That was the first thing that we set out to do. And, and this is what we found. So, two buckets.
The first bucket had four different player types. And they were a high engagement group, they played a lot of missions, which means they had action, they had fun, you know, they love the content that they engage that they saw, they played, they kept coming back. And it also, you know, meant that they spent more and then there was a second pocket. With this kind of erratic player, they weren't as engaged, they played very few missions. And of course, you know, they spent less. So, we were like, okay, we need to learn about them. So, a little bit more detail about these five player types that we found out about in the engaged in the highly engaged bucket, there was a story player, story mode, a very classic mode and games, you know, it's a curated content, where we have little bits of this kind of walking dead world, we kind of wrote our own things there, and you fight against zombies.
The other feature we had was a grind mission, where you repeat one kind of similar mission to get a certain kind of item in the end that they love. The next player that our audience segment that we found was a challenge player, where you basically join together with others to fight against zombies and get points together to get really cool rewards. And the last ones that were really highly engaged were PvP players, its players against players. And they were like, also extremely like engaged and something that we also learned at that point about the Challenge and the PvP people who interact with people, and it doesn't matter if they play against each other, or if they cooperate with other people, any kind of interaction with the real person leads to a higher engagement like that was really interesting to see.

So, these people who were in these features that set you up to work together, or play against others, they had really great fun, that was the best day they created these skills together and then played against others, like, that worked really well. But then, there was this, this one segment that didn't perform. So, we called them the sleeper. So, the sleeper was that low engagement group. So, we needed to find out why. What is happening in the experience? So, what questions can we ask? And the main thing with the data that we now had was, well, you know, maybe it's not so bad. Maybe these sleepers actually become, you know, one of the highly engaged players, they become a star player, maybe they become a challenge player. So, we wanted to find out about that. And we looked, you know, we looked into the data, how do they transition between these different player types that we just, you know, learned about, you know, and it looked really grim.

So instead of becoming one of these highly engaged players, they either stayed in this sleeper group, or worse, they churn, which in gaming means they quit your product, and they never come back, you know. So, this was the very first you know, sign of we actually have a really big problem, we need to look deeper. So, we needed to think, what else do we actually know about them by now. So, we knew that they were new players, because they had very low levels in the progression of the characters. And in the gaming content. We knew that the mission set, they mostly engaged with our story. And we knew that they would churn really quickly and leave our game.
So, the one thing again, that we had to go on here, from the data that we already gathered, is story, right? We had pretty highly engaged story players, right? This feature was performing to our best knowledge. So, going back to these player types, we were like, okay, let's do the same kind of thing with the story player, let's find out how story players transition between these player types, do they become challenge players? Grind players? Or in the worst case, scenario sleepers? Well, it turns out, that was the case. And even worse, instead of, becoming not highly engaged, they became sleepers. Or worse, they left the game.
What's going on, these kinds of highly engaged players who love the product, you know, why? Why do they leave,? And here again, you know, to show it's pretty much the same, they had exactly the same behaviour as a sleeper so now we know, okay, we have a problem. Let's, how do we go forward? And how do we do it? Now we have something to go on from the data analyst and also at any time while this kind of investigation is going on. The game designers and UX designers are usually involved in these findings and help with maybe formulating questions also regarding your product on how to find out a little bit more about the specific segments maybe of your product of users.
So what we needed to investigate, what we can learn from this is that early playoff flow is something that we should maybe take a closer look at, because it's this early kind of gameplay as the sleepers what we found, and it's the story feature that is mostly used by early game, because it is actually the very first mission feature that we are introducing, so we had something to go on. And this is the kind of point where we hand this over to the design team where, where it comes to, you know, to the game team to actually take action to take this further. And this is where I'm going to walk you through a little bit, how do we do this on a high on a high level. So now we have something we have, you know, an initial problem from data sets, we know there are these specific kind of audience segments that are not performing that they have issues when playing. We are navigating through our product, how do we find out about this, this is when we start to look into more detail, like, as I said, this early mission flow, what is actually happening there.
And this is also always a really good time to check in with your team, because, you know, not really sure how it is for you, but, you know, we really need to use our product every day, we test our product, you know, everyone in the team tests, almost every day, we have new builds, you know, with kind of new features with small updates that we are working on. And we all kind of make sure, you know, like, where are we at? What is the experience, and then on top of that, we also play our games, you know outside of this development cycle as in, in the life, you know, environment, you know, so, so the team has gone probably through the same kinds of experiences that these players have gone through. So, there are usually a lot of really good insights and learnings. And I think it's probably the same for you if you if you have tried out your own products that you are like, okay, you know, there are these various specific pain points where you're like, hey, I don't I never, I never got this really have you considered this. And this is when we also at this kind of point, we define kind of product goals.
So that means, what are the KPIs that we want to improve? And with what kind of ideas, this helps us to kind of create this design, like pillars or top level, things where we can return to, when we need to make tough decisions, you might forget, sometimes development cycles can become long, and you might maybe lose, or forget about why did you initially set out to do this, so that that really helps. So, we are still in the Investigation Phase, but we kind of start defining these kind of product goals, about what are the initial problems that we try to solve, and the main findings that we had, and we needed to take a good look at also, these underlying systems of our game, that early in this kind of story mode, for example, the progress slowed down, and that was due to difficulty rising.
And these are these really specific kinds of game mechanics that are underlying the games, how things really work just by numbers. So, they were kind of easy to tweak for us and kind of play test also, to improve. So that was really good. So, we knew we had something, okay. We found out it's really, it's, it's getting really hard early in this in the story mission to play which could be also if we wanted to have this intended experience, then there should be something else though, right, that the players can engage with. But that wasn't the case, we also found out you to looking at this kind of underlying game mechanics, and also just about play testing ourselves, that other features were actually not unlocked, you know, there was nothing else to go on, there was no other experience for you, besides being punished in this one mode, that just got way too hard.

So of course, I'm going to drop this thing, I’m not having a lot of fun. So, these were the things that we found another one to go to, that we also wanted to address was, besides that, either things were not unlocked, and things were too hard. There was something that they could do, but people actually didn't find this feature. So, we found out. And again, this is also something that I learned from actually discussing with the team, and also observing them interacting with the product, I could see how people from the development team couldn't even find certain features, I asked them to show me that the screen and they couldn't quiet, you know, get right, so, so this is a big red flag. Also, if your own team can’t navigate the product correctly, you should, you know, right away flag that.

Okay, but now we had some really main things and we were said let’s definitely address all these. And he has also a picture of what our game early, like maybe two years ago look like, you had this very basic set of navigations with only icons, when you started the game, there was this kind of camp that we know also from other games, so people need to, and this is one of these usability principles that we just didn't know better at the time people couldn't recognise them. They couldn't remember what these things actually do. And there were a lot of them.

And the other thing that we had, you know, besides this being an initial problem that people couldn't remember what these icons do. And then the second thing, if you would click on one of these to get into, you know, a really cool feature and do something cool, there were these vast deserts of texts, they were maybe not very pretty, uh, not really engaging. So besides, you know, forgetting maybe about what something does, you don't even get into, you know, a way to play anytime soon, because you are presented with some kind of, you know, layers of information that weren't maybe as important and you still needed to go, you know, press some kind of buttons to actually get to the kind of content that you wanted to actually play the game.

So those were things that when looking at the product and playing with it, that we found out, and something that we do, that I can really recommend. I know documentation is painful often. And that gets outdated really quickly, especially when you work in Agile, you know, you iterate really fast, but it doesn't need to stay up to date, all you need is the basic description, have some product goals, maybe add some user stories, later, you can add wireframes, it doesn't need to be even as much as what, what, you know, what we have put here, because this was a massive feature, the more vast or big this thing is that you do the, the more info you have to go on the better, of course, because, again, as I said, these product goals, and this, this feature brief kind of works like, you know, design pillar in the end, you know, when you create these kinds of features, you can go back later, if you have to do tough calls, Hey, why did we do this? What were the initial goals that we set out to do, and that will help you, you know, to do these tough calls at a later stage.

And then we do a lot of this, you know, it's the same as I think for many of you, we do a lot of wireframing, sketching out things, we go back then still to Photoshop nowadays, we do a lot in Sketch also. And while we do this, you know, we share a lot of these, with a team also, we have a really constant feedback loop. I know, it's not easy to share a lot of these things that you work with, and you know, they are not ready. But you know, the earlier people can, you know, point out to you what could be problematic, the better, you know, the more people, the better. I know, it's hard, you know, there's a lot of feedback, usually, but it's definitely worse. This is what we then ended up with.
So, besides all these underlying things, or the underlying, like numbers that we addressed, like decreasing the difficulty in the story, and unlocking maybe other features earlier, so, so players could interact with them, we also addressed the visual problem, and that was one of the biggest things that we did, actually, we introduced this kind of accessibility hub. So instead of these scattered icons, and with features, like hidden behind them, that do certain things, we created this one place, Hey, these are all the cool things that you can actually do in our game, they can you can all find them here, we worked on the goal setting, you can see what kind of rewards you get, you can even see when they would unlock, there are timers, and we have also other content featured then that you can play.
So, this is how we address these initial findings of people not finding things and we thought that that was in the end, also all of these things, you know, we set out to do them, but do you know that that will work? I feel like okay, we just go by our best assumptions, and by our experience, but you know, you will only know later when this of course goes live is productive, nor did you actually, did you achieve what you wanted, did it solve the problem? It's, you know, I also feel like it's hard to do these things. You go by your best assumption, and you check in with your people and, you know, you hope you hit home. So yeah, what we do, then we have this thing we call actually, we're not quite there, I'm still actually going through one round of iteration, but many of these things happen at the same time also, so what I, you know, like to do, especially when something is, you know, this was a massive feature, you know, it was a massive change to the game.
So, what we do is Team presentations, we share with the team, what is happening, we share the mock-ups, we share the product goals, you know, everyone is part of this kind of journey of understanding, you know, why are we doing this, you know, they can challenge your assumptions, they will challenge your designs, which is really a great help. If it's smaller things, you can just share them in the Slack channel also. I think that's really helpful. Because this is the initial reaction, at least from our kind of teams, they have a lot of questions, which are usually really good things that you haven't just thought about, and there's always something you haven't thought about and then it turns out that they see what you're doing, and they think it does something completely different, it really helps a great, a great lot.
So you do a lot of this kind of iteration with the team, you keep sharing the iterations, the team stays involved at any time, you will never get to that everyone is happy, but you kind of will get to the point where you feel like, okay, I have addressed the biggest issues that people have pointed out, you know, and the rest needs to be verified, you know, when maybe the product goes live, then there is another tool that we are using, to make sure that these kinds of features that we have, you know, so meticulously planned out, really make it into the game in a certain kind of quality, it's called a feature team. So instead of back in the days, we kind of picked people from the team, who also made features, but also a couple of other things, we now have an, a team that actually takes the ownership, they are doing this one thing, they have the complete responsibility of shipping this feature, and also iterating on it while developing, you know, they do the testing, everything they make sure that we have set out with the design team and with the team in general, that that was also what ends up in the game. Because usually, you build something and there are also more learnings, right? So, you want to have to have those things in there, then they perform something called a component breakdown.
So that is kind of just the functional spec. Okay, what kind of buttons do we need? What kind of logic is necessary for all these screens? From that kind of thing? You know, we make sure, okay, is this feasible in what amount of time all of these things are really important, and, you know, the more ownership the people in your team have, the more you know, and they will struggle on the start, but the better that people can learn if it's their responsibility to do these things. And then we do the same, I think as many of you, you know, we implement in Agile, we iterate a lot, we do a lot of bug fixing, we polish we, we check in with the other teams in house, we maybe let your mother play or your, you know, your friends, I think that really helps. And then it all goes through QA. And if you remember, this is what initially, our game looked like with this kind of, I can set people that know what kind of hides behind them. We added Finally, these kinds of texts with them.
So, no one will ever forget what any of these are about. And the other thing, if you have seen there was one feature in the bottom right? That was leading to this kind of story missions, we now instead have this kind of hub, where we have collected all of them. And, you know, initially it looked like this, and this is how it goes, you set out to do the same. And this is what's actually in the game. So, you know, a lot of things, you know, are happening from this initial, you know, draft and the initial thing that you set out to address, but there's usually a lot of business, things that come also there that are important, also feedback, and maybe we want to have some kind of ads there. And also, we need some time. So, this is how things change throughout this kind of development cycle, and also being live and getting feedback usually. And you remember when you opened a feature, you had this unlikable kind of screen with all kinds of inputs in them.

So now instead, when you click on one of these tabs, you are right away into the kind of feature. This is how all the missions look like any kind of feature with the mission that you will interact with, has the same kind of look. So, you can right away actually start and get into the action of these kinds of features. So, there's fewer things in between, it eases the flow to some of these things.
And yeah, the big release, right, it's there, I have to say, the big learnings from this kind of stage is allocate time for not only to review the data again, but, you know, for us, due to the nature of games and that we work with franchises, we have a lot of feedback you know about what people love or hate, you know, there they are, they are extremely opinionated and engaged, you know, so we actually need to allocate a lot of time At least one week or more to come back to what have we done? And what is the perceived experience for all of these players.

So besides them writing about this, there's of course, the new kind of data that comes in, that basically shows these assumptions that you made. Are these correct or not? Yes, I need to speed up I can see, okay, and you know, and in games, it's really hard, you have this, you again, you set out to do something you try to verify with your team, we find out there sometimes you did is confusing things that you left out that you thought were okay, in the end, they are missing, you need to allocate time to bring these back.

So, what did we do, we monitored, you know, besides your daily data that you kind of receive, try to find out a little, make some snapshots, learn a bit more about these users that you are catering towards? How do their journeys look from day to day, from week to week, in very specific segments, you can learn a lot about this. A proper investigation involves everyone you know, there are great learnings among a team everyone has, who works on your products, you know, them, they might have really good ideas, and just involve them.
And then in the end, also test with them and keep iterating. And my biggest takeaways are, create really strong relationships between your analysts and the game designers or UX designers. There's so much potential there. It's really incredible. And, I think it can always be improved, because often they might be too, working too siloed. And also, don't slack on the documentation. You know, I know it's painful and it doesn't stay up to date. But it really helps you in the long run to see that you would also, did we design something that in the start what we set out to do, and give your team's full ownership. There is no loss there. And that's all. Thank you.