Designing With Player Segmentation Data At Next Games

Jasmin Verollet-Dahncke at UXDX Community: Helsinki. Video: https://youtu.be/227RaT9F8FE

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Introduction and Next Games

[00:00:08] I'm from Next Games, and today I will be talking to you about designing with player segmentation data in our game, The Walking Dead: No Man's Land. It will be a really quick speedrun, and we'll look at it from a top-level view, because I'm not a data analyst. But I will show you roughly what player segmentation is and how we do it, then I'll show you how we design, and a little bit about our feature pipeline: how we make a feature based on what we found there. So let's get started.

[00:00:45] The agenda: let's start with a quick introduction. I'll tell you a bit about who I am and who Next Games is. Then I'll tell you a little bit about games as a service, because that's the business model we are working with, and then we'll get to the segmentation and the feature development pipeline.

[00:01:03] My name is Jasmin. I'm a UX designer at Next Games. I have a background mainly in graphic design, and I've always worked in the games industry. I worked a lot on merchandise; I did a lot of magazines and book packs back in the day. Then about four years ago I entered the game industry as a UI artist and made the transition to UX designer, because information hierarchy is really where my heart is: creating an experience that is understandable and enjoyable to our players. That's what I do.

[00:01:41] The company I work for is called Next Games. We mainly make games for really big franchises, so we work really closely with license holders to bring these TV shows, cinema movies or books into a game experience for the fans. Here you see some of our games. One of the first games we did was The Walking Dead: No Man's Land. It's an RPG based on The Walking Dead franchise. We are actually making another game with this franchise; this time it's a Pokémon Go kind of game, a location-based game, so that's something really exciting. One of the other IPs we currently have is Blade Runner, but not much more has been announced about that. That's something we are working on.

[00:02:31] But the game that I'm going to talk to you about is The Walking Dead: No Man's Land. I'm thinking that this video might not work; I really wanted to show it to you. Okay, that's fine. How many of you actually play games? Could you raise your hand, so I can see how much I need to explain this? Quite a few of you play. This is a strategy game where you drag your characters in a turn-based manner towards zombies and fight them. Because it's turn-based, you have a lot of time to think, how am I going to do this move, and then you figure out how to fight them. Because it's based on the IP, we have a lot of tie-ins with the show. You can revisit locations that you have seen on TV, you can fight the same bad guys, like Negan, or you can fight as Rick. That's what it's all about.

Games as a service

[00:03:27] Games as a service: I want to give you an introduction to this because it's really important to why we do segmentation, and it's based on this business model. It's called games as a service. It's not really new; I think every one of you knows free-to-play, for example. It's a continuous revenue model. It's really different from how games were classically made. You had a really long pipeline of a couple of years, you made a thing, and then it came out. For example, God of War: it took five years to make, and then you purchase it for 60 euro. They have made an initial sale, and then they can kind of forget about it. Nowadays even a game like that gets patched, but back in the day that wasn't the case.

[00:04:14] Nowadays, with mobile phones and the PC in your pocket, games have evolved to these kinds of devices, and one of the revenue models that came with them is free-to-play, games as a service, which really took off, because it's really easy for people to try a product if it's free and find out if they like it or not. Something that usually comes with these kinds of products is that it's a live product with continuous development, and you usually want to have a really long life cycle to pay back the initial development costs.

[00:04:53] Something about the key performance indicators for this kind of business: this is how we measure our success. Why are these three KPIs important? Because our product is free initially, no sale has been made, as in the classic video game business. We really need people to come back. We need to retain them, through engagement also, and we need to convert them at a later time. Not through an initial sale, but they come back and make a purchase at some point, based on something they had fun with, maybe a cosmetic, or some other game-related item that they wanted to get.

[00:05:41] Something else I really wanted to say regarding this business: I guess most of you make apps that solve a problem. For example, Yousician makes an app where I can learn an instrument, or there are apps that book me a cab. We are making a game; we are not really solving a problem for people. So it's also much easier to drop something like a game. It's pure entertainment. It only entertains you: you can have some fun by playing, but it does nothing else in that sense. You can quickly forget about it and uninstall it if it doesn't work, and there's no backlash for you. That's important to understand when making a game, I think.

What player segmentation is

[00:06:30] Let's get to player segmentation, which is related to how we get there. We have these KPIs, we have this business model where people don't need to pay initially, and we have these KPIs, retention, engagement and conversion, that need to work so that our business can be successful. How can we design with these in mind, or around them? That's where this whole segmentation thing comes in. We have a lot of data analysts who help us, who do initial research for us and find things regarding these KPIs that we can improve.

[00:07:08] What is segmentation? It's basically about finding out who your audience is, who's using your product, and where they are spending time in your product. Our game, for example, is really big and has a lot of features, so really seeing from your data where they are, what they're doing and how they are progressing is quite important. That's what this whole segmentation thing is. I think most of you have made the transition to being really data-driven companies, and it's the same for the gaming industry.

[00:07:43] How can we identify different segments in this game audience? We took a really basic approach. This was, I think, two years ago now. We were thinking, what missions do our players actually play? Let's make segments based on this, because we have a lot of different missions in our game, since we have so many different features for fighting zombies in all kinds of ways. So, okay, let's make segments based on the missions they're doing.

[00:08:14] Additionally, we can ask other questions. You have these audience segments, or player profiles, as we also call them. How do they actually progress in relation to the segments? People age, or mature, throughout your product life cycle the longer they stay in there, so they're not always going to stay in the same segment. Sometimes they move from beginner to a more advanced, or elder, player, as we call it, and they do different things. For us, as you progress with your avatar or you play a level, other features become available to you that will be much more interesting, for example. That's what this is all about: finding out how they progress and transition.

[00:09:03] And of course it's also important for us to know how the segments are spending, because it's free-to-play and we really need to understand that. This also comes from a place of not just monetizing as well as possible no matter what. It's about finding the fun and the engagement, so that you value this kind of virtual coin, and it has an inherent value for you to buy it, and you feel good about yourself. You're like, hey, I bought this cool skin, it makes me look really cool, and I really enjoy myself. That's what this is about.

The sleepers

[00:09:41] This is what our data analysts did. They took one week of data, basically a snapshot of our game in that one week. The main measure was: what missions do these people play? They use a lot of other techniques, like clustering and stuff; I don't really know about that, and that's a whole other presentation. It's crazy what you can do there. They are so smart, I can't even understand it, but it's really cool stuff that we get from them, what they read from our app.

[00:10:15] This is what they found: five basic player types. Story players and challenge players; the names are based on our features. Story players: there's a story mode, where we have our own part of The Walking Dead world and have written little stories about it. Challenge players: this is a multiplayer feature where, with your friends and your guilds, you accumulate points on specifically designed zombie maps. Grind players: again another feature, where you're grinding away in one specific mission that has very specific drops that you're after. And PvP, the player-versus-player mode, where people fight against each other.

[00:11:05] What they found is that these four are super engaged. They play five to twenty missions, and they spend pretty well. All the right KPIs, retention, engagement, conversion, are working well. And then they also found an outlier. They called them the sleepers. We were like, what is going on? They're not spending a lot, and they're also dead because they don't have fun [?]. They're playing one to two missions a day. Okay, maybe we should investigate what's happening and why these guys have this erratic kind of behavior of not logging in or playing so much.

[00:11:51] When we looked a little bit more, and here you see a transition, what we asked ourselves was: how does this player transition throughout their life? Do they actually become one of the story players, or one of the others who are heavily engaged? It turned out that no, either they stay the same, at the same level of one to two missions a day or every couple of days, or, even worse, they churn, and in really big amounts. That was an alarm signal. We were like, okay, we need to understand a little bit more about that.

[00:12:37] Something we knew about them from the player types was that they mostly play story. The one reference we had was story players. We had checked how the sleepers transition through the player profiles, so let's check whether there's anything different in the life cycle of these story players and how they transition through the profiles. It turned out, and it was scary, that there was almost no difference. They transitioned exactly the same. And these were really engaged players; we saw they were playing a lot every day, five to twenty missions. And they were also turning into either sleepers or churned players, never coming back. So we knew something was up with our stories, and we needed to take a look.

[00:13:35] What we found when we took a deeper look at what was happening is that story is the first feature we introduce in our product, so it's very early game, mostly very young players, and it's really difficult for them to do anything else. First, the difficulty rises, so playing this feature quickly isn't so much fun anymore, because you need to level up a lot. But we didn't unlock other features in time for them to engage with. They couldn't really do anything else. That's why we figured out that we needed to do something about it. The accessibility was bad, and the transition to other features just wasn't there. That's what we found.

[00:14:29] Those are basically the findings from this whole segmentation. We now had a really distinct problem, or a question or challenge, that we could pose to design, that we could give to the game designers and the UX designers. We had all these transitions that we could show: hey, we've found that story difficulty rises and other features are locked. This then goes to the team, so the dev team can take this problem and act on it.

The feature development pipeline

[00:15:00] That's what happens next: the feature development pipeline. Now they come and talk to us. Usually we are already involved all the time with what's going on in the research, but now we get a proper onboarding: hey, we have found this thing. Data analysts, game designers and UX are discussing what the problems have been, and now it's our job to find out how we can make this better for these sleepers, and how we can relieve the problems they have been facing, as found in the research we just did.

[00:15:38] There's a lot going on here that is not on the slide. You check in with your team. That's something I've heard a lot in the other talks, and I can just repeat it again: check in with your team. Most commonly they already know what's up. They've had ideas already. They have experienced the same pain points, because they have played the product a lot or interacted with it a lot, and they might have really good ideas. That's what happened here. I checked in with the team, and they were like, yeah, we have this idea, we've known about this problem for a long time. Because it's not only early-game players. I had also seen my colleagues having problems navigating the game or finding certain features, so there were other problems that would also feed into this.

[00:16:34] What we decided to do was a mission hub: a central access point that would help guide players to the different features and show what they would get, the rewards. In conjunction with that, we would need to balance the difficulty and the unlocks of the other features, which is more on the game designers' table, so that there's a better flow from story to challenge or grind or what have you. They did a lot of things there, which is again a whole other presentation on game design.

[00:17:12] This is how it goes forward. We find ideas, and when we have an idea we start to sketch something out. You do a whole lot of this. This was the start of the mission hub: how should we present all this information? What should be on the screen? You do a lot of this, and then a lot more of that: you do the mock-ups for the game. At every point I'm checking in with my team, sharing these screens with them, and letting them know the things I've learned or the problems I'm facing. I feel like my job is so much easier; I feel like I let them work for me, but it just helps so much to get a different perspective.

[00:17:56] What we then decided to do looked like this. We have this mission hub, with our cool content featured here. Here you see your story, and you see the rewards. Now you can actually see all of our features laid out. Unfortunately I haven't added a screen of how it was before, but there was nothing. There were only icons in the main hub where you were playing, and there were no names. You didn't know where you were going. You had a camp, a little bit like Clash of Clans, where you collect resources and build buildings, and you didn't really know what the other things do, and you might forget. Now you had a central access point with all the features and all the rewards. It also said when they unlock, with timers and what have you.

[00:18:56] Again, at this point you've written something up and you introduce it to your team. For me, they're always my gating process. Do they like it? Do they think I'm on the right track to solve the problem? Usually they're like, what is this? Have you thought about this? Why did you do this? And you just keep iterating until, maybe not everyone is happy, but you feel like this really addresses the problem that we set out to solve when we started.

Feature briefs and feature teams

[00:19:31] Then we do something we call a feature brief, and I really recommend this; it's pretty good. You sit down, now that you have a mock-up and an initial draft of how this should work, and you write down the product goals again. You write down the user stories, and partly the components of how it works. It's a point of reference. The thing is, we do agile, and things change a lot, so it doesn't keep up to date, but it should capture the initial idea of why you did this thing. There can be the research from the initial segmentation. The product goal should be there; even if you just have a small description, I think it's enough. Dump the screens there. Then there's something you can share. We also have a lot of other disciplines in the company, like player support or marketing, that need a point of reference, so there's something they can check. Of course they still need to check in with the team to see whether something has been updated or not, but overall it really helps. I also think the moment you write it down, you might find things that you actually haven't thought about, so it's really helpful.

[00:20:48] Then production can start. That's what we saw earlier in the other talk: that's when the sprint starts. You have a design spec ready, so people can pick it up and create something based on it. We form something called a feature team, where there's someone from every discipline: back-end programmer, client programmer, UI artist, game designer, UX. These people are responsible for making this thing. It's all on their table. They're going to make the decisions, they're going to do every additional iteration, they're going to do the user tests. For us at least, it really helped. We had problems earlier in development with who's responsible for this thing and who does it, so defining a team of people who do this thing really helped keep things moving forward.

[00:21:49] I skipped over something: they do a component breakdown. At this point they also need to look at the feature brief, for example, and ask, to make this thing, what do we need? You check all the buttons; you need to go through every single piece to see how we build this. For features, you have an MVP, a minimum viable product, but you also have something maximal, and half the features won't make it in. That's just how it goes. So they need to do an initial draft of what really makes it in, what's essential to this game experience, and what we might add later, or never, which is usually how it goes.

Implementing, testing and polish

[00:22:35] Then you implement and iterate, agile. You've seen this before, but it's not how it will end. There's still a lot of iteration, and we do a lot of user testing. What I do, for example, whenever we have something that works in a build, is a small user test at my company. I invite the whole company, see who shows up in the kitchen, and we do half an hour. You're just there to observe and to ask some questions.

[00:23:08] Something we also needed to learn throughout this process is: never explain your design to anyone. I feel that's my biggest lesson of the last years. It's so easy, when someone is showing you a thing, to say, yeah, but this is how it works. But no one will be sitting next to your users telling them how it works. You just have to sit there and be like, mm-hmm, yes, I will write this down. It's a really tough lesson, and it sounds easier than it is: not answering people who seek help is tough. Then you need to go back to your team and discuss the things you found that don't seem to be so easy for people to understand.

[00:23:51] A lot of bug fixing, and polish if you have time. We never had time for polish, for example, but nowadays, from the production side, we have allocated one week after every sprint just for that. With these features, you still implement until the last day, when there's a freeze. But at least for us, a feature freeze doesn't mean it can go out like that. There's usually always some tweaking, some missing feedback, some animation or what have you. I have to say it really helps to allocate a little bit more time to polish before QA. It can also run at the same time as QA, but it helps to have a bit more.

[00:24:42] This is what we set out to do: we wanted this cool accessibility hub, we tweaked the transition flows, and we unlocked features earlier. And what we ended up with, what's currently in the game, is this. This is how it goes with iteration and agile. You thought you'd make this thing, and it kind of is still that thing, but a lot has changed based on business needs or on user feedback. That's where we are now.

After release, and takeaways

[00:25:15] Then you have the thing, you release, and it's not over. You need to wait for the player reaction. For us, I do this kitchen kind of test with a small sample, and if we're lucky we have some playtest cloud, a bigger sample where we can test, but not always. Now is the time when you actually see whether this thing works. In the past we didn't really allocate time to react to what we found after the initial release, but nowadays we have someone for one week after each update who fixes stuff and also looks at whether the thing we decided to do is working, to a certain extent, for people. Can they understand it? You go back, you fix things, you add the missing feedback that you thought wasn't necessary for the MVP, and then you find out, yes, it was necessary, so let's add it. And for games, what comes up a lot is mechanics: how certain things work under the hood, how people progress, what I need to collect to do the next thing. That's not always super clear either, so we need to look at that.

[00:26:35] And of course, if you have time, you review the data. We did this feature because we did data research and found a problem. For this one, for example, we never had time to go back and review whether it did something. Again, that's just how it goes. If you can allocate the time, I definitely recommend it. Nowadays we have so many other things in the game that it's also really hard to measure. We have so many features and so much new stuff in there that you don't really know anymore what the one thing was that tipped the scales.

[00:27:14] So this is how we work. You learn about your audience. You segment the data. Ask the data guys in your company to work with you, and let them explain what they're doing. Maybe you can find pain points, and then you can react to them. You design solutions, create mock-ups, and test with your team: does this alleviate the problem? You implement, you iterate and you test, and that's really it.

[00:27:39] Data is everything; that's my takeaway for you. If you have a data analyst in your company, you're already really far ahead. Go and talk to that person. These guys are awesome. They can help you learn so much about the people you make stuff for; it's really crazy. Second, a feature brief: writing down what you've decided to do helps you understand it more and maybe find flaws in what you've set out to do. And more power to the team, with feature teams, and asking the team, hey, we want to do this thing, what do you think? This isn't just the thing you're working on. Maybe it's stronger in games, I don't know, but this is not my CEO's product or my marketing product; we are all doing this thing together. So trust yourself, and trust the teams too, that they know what they are doing. That's it, thank you.