Integrating AI into the Game: King Games' Journey of Technological Transformation

Steve Collins at UXDX EMEA. Video: https://youtu.be/GMaLsDLSatE

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

King and the mobile games industry

[00:00:01] Great to see everybody, and I'm really delighted to be able to talk to you a little bit about our AI journey at King. I'm going to get straight into it. I've got quite a bit of content and I'd like to leave some time for questions.

[00:00:12] You might be aware that the mobile game industry is pretty large, but I just wanted to set some context for you. It's the largest portion of the gaming industry, about 53% overall of games today. The gaming industry itself is worth coming up to $190 billion [?] worldwide, which is quite extraordinary, and mobile is more than half of that. It started way back with smartphones and Snake and lots of very simple games, and it's got to the sophisticated, nearly console-quality games we have today.

[00:00:42] King's journey started quite some time ago. The company is 20 years old this year, and Candy Crush was released just over 11 years ago. We started out creating skill-based games on browsers and very quickly pivoted to releasing games on Facebook, so the growth of the company was very much in parallel with Facebook's meteoric growth. Some of our earlier titles on that platform include Bubble Witch and Pet Rescue and things like that. Around this time we introduced what we call the Saga progression mechanic, which I'll talk about in a second. Candy itself was launched in 2012, as the whole mobile ecosystem was exploding around the app stores and the iPhones. Since 2012 Candy has just grown and grown and grown, and it's been an incredible project and entertainment franchise to be part of. It has been the top franchise in the US for 24 quarters in a row.

Candy Crush in numbers

[00:01:42] A little bit more about Candy. Hopefully some of you here will have played Candy and experienced it. We've got some fun stats here. It was launched in 2012 and it now has over 15,000 levels. If you know Candy, you know at its core it's a very simple puzzle switcher game, where you complete matches to progress through the level. With 15,000 levels, we have many thousands of players who have played all the way through our game and are waiting for the next drop of content. Every week we drop more levels, another set of episodes for our players, and that becomes their weekly routine, which is really incredible. Levels and level content are at the core of our business, and it's going to be a topic for AI in a few minutes. We've also had over 5 billion downloads of the Candy franchise. Right now we've got over 238 million monthly active users across our games, and something like five trillion levels have been completed. Can you believe it?

[00:02:41] What is Candy? Candy is at its heart a very simple game. It's a casual puzzle title on mobile. It's free to play, so there's absolutely no requirement to pay to play Candy. It does of course have an economy behind it. We are a commercial company, and there are two ways that we generate revenue. One is where some players, a small fraction of players, choose to pay to get boosters and lives and progress faster. Then another portion of our players watch ads embedded in the game. These are opted-in ads, so you absolutely never get an ad if you don't ask for it specifically, and in return for watching an ad you get rewarded with boosters and extra lives so you can progress. Our job is essentially to balance the game experience for all these different types of players.

An entertainment platform, not just a puzzle game

[00:03:25] Right at the core we have our switcher, which is essentially the simple match-three puzzle game. Around that we've built the idea of a narrative, a progression over time, so that players get a sense of achievement. It also gives us a way to tell a story, to bring players on something of an emotional journey. It's not just that the levels get increasingly hard. We're trying to create fun and excitement for players so they really feel engaged with the game.

[00:03:54] A third way we think about these games is as an entertainment platform in and of itself. We've got loads of players in Candy. They come to play the game, clearly, but some players also come to compete with their friends, to set up guilds [?]. Some players are very competition-oriented; they want to achieve high rankings in competitions. Other players are very much interested in a solo experience. They've just got a couple of minutes in their daily lives and they want to zone out and get some delight and some fun. And then we've got players who are very social, who just want to share with their friends and their community and build small communities around the Candy game. We're trying to allow all those different types of players to interact and experience the game in the way that they choose, so it becomes very much an entertainment platform, more so than a simple puzzle game.

[00:04:40] To give you an idea of the complexity of some of the things we do inside Candy as a platform, we have increasingly been running competitions to try to find the number one Candy player. We started out in the UK, then expanded to the US, and last year we had the first global Candy competition to try to find the top Candy player around the world. It's such a fun thing to do, to have millions and millions of players competing for a cash prize, all the way down to the ten top players, who we then invited back to our London HQ for this wonderful, fun event. That requires a huge amount of coordination, quite a high investment in technology, as you'd imagine, and quite a lot of work around branding and co-branding and working with partners to bring those players into that competitive environment and get them to have fun. So it's way more than just matching three candies in a game.

[00:05:33] That gives you a sense of how we think about Candy as a platform, as a technology and an entertainment experience. I want to give you a sense of how we intersect AI with that, and how we've increasingly been using AI to create, to operate and to optimize the Candy business.

A short history of AI in games

[00:05:50] I'm going to start by going back in time a little bit. AI and games have had a very interwoven history. AI has arguably been present in the very earliest games. It might be stretching a point to say that Pac-Man has AI in it, but it has rule-based non-player characters, NPCs: Inky, Blinky, Pinky and Clyde, the four ghosts that chase you. Each one has a slightly different personality. The personality is just expressed in terms of how they track the player's movement, but they're arguably the first rule-based non-player characters, essentially an AI.

[00:06:28] Moving forward a little bit, I'm a big fan of the Commodore 64 as well as the Spectrum. On the Commodore 64, David Crane from Activision, who was famous for Pitfall, also created this game, which is less well known. This is Little Computer People, created in 1987. LCP was like an early version of a Tamagotchi. You had a single character, an AI character that you controlled. You'd instruct it to go to bed and wake up and eat and things like that, and essentially you nurtured the health of this little character. It had an AI behind it, so it's a nice early example from the home computer era of the 80s.

[00:07:10] Those of you who are gamers might recognize Black & White from Lionhead, a UK company, created by Peter Molyneux. This was a god sim game where you were trying to essentially evolve society forward, and at your disposal was this large creature you see in the background. The creature essentially learned to do your bidding and could become good or evil, thus the name Black & White, based on how you incentivized it. What was behind that creature was the first example of reinforcement learning in computer games, at least the first recorded example. Reinforcement learning is now very much part of the fabric of AI as we use it in different parts of our business. Nice example.

[00:07:49] The last example is the AI director in Left 4 Dead, a title from Valve. This really brought to the gaming community the idea of an adaptive game, a game that's trying to invoke a sense of tension and an emotional response from you. It's messing with you as the game progresses. The AI director decides exactly how many enemies to keep pushing towards you. It decides exactly how your cooperative players, who are working with you and controlled by AIs, behave, to try to create this tension. They keep passing waves of enemies towards you until you just about run out of bullets, and only then do they stop. You always get that sense of tension, no matter how good you are at playing the game. So again, AI embedded in a game.

Generative AI and what it means for King

[00:08:36] Now AI has clearly gone through some amazing transformations in the last while, with generative AI and large language models, and you heard from Fergal [?] just before this on how Intercom are using LLMs for Fin. This is going to have a big impact on gaming. At the moment most of the gaming industry is trying to figure this out. We're trying to figure out how we can engage with things like generative content, how we can do that in a safe way and in a way that's respectful of ownership and copyright. I think there are a lot of questions still to be answered there before companies like King can truly engage with that.

[00:09:13] But there are some really nice examples in the literature. This is a paper from Stanford from last year which showed the art of the possible. Imagine taking something like ChatGPT, which you can have a perfectly valid conversation with, but, as you probably know, with a bit of prompting you can get it to have a persona. If we attached an instance of ChatGPT to the non-player characters in a game, and this is a Zelda-like Sims game, each of these characters has its own ChatGPT instance that's essentially making it decide what to do next. You give each character a different personality, a context and a set of goals, and then you stand back and watch it play forward. It turned out to be quite boring in the end, but it's really interesting and it's worth having a look, because it's available online and you can tweak the personalities of the agents to see how it progresses.

[00:10:02] For us, we're very excited about this. There's a huge amount of potential offered by generative AI in general, from enhancing game features, to enhancing how we produce our games, to how we think about software development. That's the thing I'm very excited about, just the art of writing software. And finally, as a general cognitive productivity aid. I'm not saying anything here that people aren't fully aware of, but for us, we don't think for our types of games that this is going to have a huge impact on our game features. It's certainly going to impact how we produce our games, how we test our games and how we operate them live.

[00:10:40] There's no question that software development is changing, and we're looking to embrace that. We've run pilots that match some of the industry stats out there about productivity enhancements from using generative AI, using large language models to aid the coding process. But this is the area I think we're most excited about: can we connect this productivity tool, this cognitive enhancement agent, into how every craft at King does its business? We're running that experiment at large in King right now.

[00:11:12] We've had a long journey of AI evolution in King, starting from 2016, when we started to look at the challenge of creating levels, because increasingly that was becoming our bottleneck. Through the years we've increasingly been looking at different areas like personalization and recommendation engines. Around 2020 we published an AI strategy. In '22 we made an acquisition to accelerate our work in this. Right now we're in the middle of, A, looking at how we use traditional AI techniques to improve our business, and B, looking at what this explosion of new capability and technology means for us in the next three to five years.

Recommendations and reinforcement learning

[00:11:52] I'm going to give you three use cases, to give you a sense of how we're using more traditional AI. We're not going to talk about large language models here, because we're still very early in examining that opportunity, but there's a whole history of AI that we can take advantage of as well. Here's what we do. This is our business. I don't expect you to follow it; it's actually a bit of nonsense as well, but it gives you a sense of the complexity. A game like Candy Crush, or indeed a business like King, in some sense represents the perfectly observable business model, because everything is online. We can track everything that our players do. We know when players are spending, we know when players are leaving our games. In a sense it's a complex, highly nonlinear dynamic system, and it's our job as creators of that business to optimize this highly nonlinear dynamic system.

[00:12:42] But you have to break it down. At the core of our business is the core loop of the player playing levels, finishing levels, maybe not completing a level, maybe getting frustrated, maybe feeling like they're making incredible progress. At different points in that level loop we get to have a conversation with them. We get to invite them to do different things, for example play with their friends or join a competition, dot dot dot. We get these points in time, about once every minute or two, to have that choice presented to the player. But there's an overwhelming choice of things that we can do for that player, and the last thing we want to do is get in front of them and introduce friction into how they're playing the game. We want to get out of their way.

[00:13:23] So increasingly we're using recommendation technology, which over time learns what different players are trying to achieve and optimizes specifically for those cases. Essentially we get away from the tyranny of choice, presenting a player with every option, and we try to hone it so that we get them to fun in a much more efficient way. Each time you make that choice, it's not in isolation. It depends on what's happened before and it will certainly impact what comes after, and this is where it becomes highly nonlinear from a temporal perspective. Not only do we have to think about the choice we make at any given point in time for a player, we have to think about the sequence of choices we're making on behalf of the player and how we think about that progression over time. We're optimizing for time, not for the instant. This is a great use case for reinforcement learning, where you're evolving and optimizing policies for players over time and adapting to individual player behavior.

AI players that give level designers feedback

[00:14:19] I mentioned levels are at the core of what we do, and they absolutely are. We generate thousands of levels every year, in fact tens of thousands in some cases. Our levels are really important for us, and it's important that they're something we have full control over. Luckily, as a data-driven organization, we have a huge amount of data at our disposal. We use that data to feed back and say whether levels are working for us or not. We look at the number of attempts per success that a player has, we look at the number of shuffles, times when levels haven't been possible to complete, and we use that to feed back to our level designers.

[00:14:51] Increasingly, what we're doing now is developing AIs that act as players. AI players, not to beat the players. We don't want to be better than a player, we want to behave like a player. And there's no average player, so we're looking at AIs that behave like different types of players, like a player who's quite happy to pay and is using boosters, and so maybe has an advantage in their ability to progress quickly through different levels. We're optimizing these so that we can provide feedback to our level designers. We build models, we look at all this data, and over time we build a system that allows our level designers to learn very quickly whether a level is difficult or not, or going to be fun or not. As our designers are creating the levels, we're providing immediate, nearly real-time feedback to them, to say that change you're making is going to make the level 10% harder or 10% easier. That gives our designers huge agency over the experience that they're trying to create.

Predicting lifetime value for performance marketing

[00:15:48] The third example is performance marketing. You may not be aware, but in the mobile game industry the main business loop is advertising to players to bring them into your game. Then those players spend time with you, either spending directly or watching ads or just contributing to the overall success of the game. Over time, some of those players, in fact the majority of them, will eventually leave and move on to other games, hopefully other King games. That's a core loop for us that we need to optimize.

[00:16:15] This is what it looks like. We work with a lot of partners, on the left-hand side, through which we advertise to our players, particularly on mobile channels. We can tell them that Candy has a new piece of content available, or a new experience, or advertise any of our other games. Those players come into our game via the app stores or by being re-engaged, and then they start playing. As we advertise to players we're spending money on the advertising campaign. Imagine we get a thousand players in from one given campaign. We have to make sure that the amount we spend on that campaign is less than the amount that those players will ultimately pay inside our game in aggregate. The formula on the bottom is our recipe for success. We have a return on investment if the lifetime value of the players that we bring into our games is higher than the overall user acquisition cost, or UA cost. That's what we're optimizing for.

[00:17:09] But how can you do that? You're acquiring a user now for a certain number of dollars, and you're doing that on the basis of an expectation of what they're going to do over the next 18 to 24 months. What you try to do, in as short a space of time as possible, for a group of players, is predict what that lifetime value is going to look like. That's exactly what we do over on the right-hand side. We make a prediction, and if that prediction says yes, we think that group of players we've just brought into the game is going to spend money in time, and a lot of them aren't, of course, but many of them will, then this is a good channel for us, a good message, a good campaign, a good partner, and we double down on that. Getting that decision window as small as possible is really important, and that's another area where we spend a lot of time in our AI, optimizing those LTV predictions. I should have clicked through to "predict," and it loops around.

Landing AI in the organization

[00:18:01] How do we go about doing all this? Those are a lot of examples, and maybe an abstraction of how AI might be used, but bringing AI into your business is really hard. Particularly, I would say, for businesses that have adopted a very specific way of working. King has a very excellent way of working, very data-driven, but then you're trying to think how to alter that way of working to take more advantage of automation and automated decision-making. This is a survey from IBM from a couple of years ago on the classic impediments to bringing AI into your business. Data is the top one. That wasn't our problem, but some of the other problems were definitely reflected.

[00:18:39] Here are some specifics of King. At the initial state, when we started doing this, we had a very mature data infrastructure, but the actual AI implementation tended to be sporadic, in different pockets. There was no single path to actually getting AI into production and getting to the point where it was generating value. It was all special projects, essentially individual use cases. There was no clear ownership or governance, and particularly as you start to leverage more and more of this technology, you have to have a way to govern it inside your business. And there was a limited number of applications of AI and machine learning, and what we wanted to do was ramp it up completely.

[00:19:18] We also recognized that one of the restrictions on doing that is access to talent. Getting people who are experienced in AI and machine learning is an incredibly competitive exercise right now, so we made the decision that we would acquire to help us with that. We acquired a company in Sweden called Peltarion, who were experts at deploying AI and machine learning technology into large enterprises. That immediately gave us an injection of 45 people who had experience doing exactly these sorts of things.

[00:19:45] Then we started doing exactly what companies like us should be doing, which is thinking about how we land this capability inside our organization. You don't just open a web page internally and say, "Here is the AI, come and get it and make your business better." You have to be really thoughtful about infiltrating your business and essentially disrupting the current way of working. To do that we created an AI Center of Excellence, a team that's focused specifically on working with all the different business units in King to identify opportunities. To date we've identified over a hundred different use cases that we think could have million-dollar-plus impacts on the business.

[00:20:22] We need to create awareness of what's happening in the AI space in general, and awareness of how different crafts might take advantage of AI. So we created our AI enablement team, which is essentially training. We're training our executives, our data engineers and our data scientists in the latest technologies that are available, and you can imagine that's getting pretty busy lately, with all the stuff that's happening in generative AI and large language models. And finally, we're also investing in our labs: looking forward, what are the things coming down the line that are going to impact our business and enable us into the future?

The AI maturity curve

[00:20:53] We're at a point on an AI maturity adoption curve that's very classic in a sense. Just to nearly finish off: you start off essentially with experimentation, trying to figure out the art of the possible, learning and seeing what's out there and what might hit your business. Then you start to do things like we're doing right now: build up an internal consulting organization to aid each of the business units in thinking about how to use AI, essentially providing professional services internally, and start identifying a couple of use cases where you can show early value.

[00:21:27] Then you get to the next phase, where you're trying to get repeatability, and you look at use cases where, if we just build an entire team around this, it's always going to generate evergreen value for our business. A good example of that might be our level creation, so we have a team that we're spinning up specifically around AI for level creation. The next phase, and this is probably where we are in our maturity journey, is where you start connecting the dots of all these systems. You start having AI as nearly a default case in all the different planning you're doing, connecting the dots into a single fabric with AI enabling every part of the business. And finally you get to maturity, where every time you're thinking about something in the business, AI is one of the first things you think about. I'd say we're at that level in data maturity. We're an incredibly data-mature organization, and now we want to catch up and get to that point on the AI side.

Lessons, warts and all

[00:22:19] I'm going to leave you with some lessons, warts and all, because it's not nearly as fun and easy as these colorful slides generally suggest. Anybody who's worked in large companies knows there are frictions. Trying to make change triggers antibodies in the organism. Here are a couple of observations.

[00:22:38] First, there's huge power if you can get the company to coalesce around a single objective and a set of key results that enable this to happen, because it cuts through a lot of the decisions that need to happen in your business. This year we have an objective at King company level to drive AI in our business, and that objective has real revenue numbers attached to it. That's great, because at any given point in time there's always friction. You have teams who are doing what they do, generating a huge amount of revenue, being really profitable, and you're trying to tell them, let's do something else. That something else might not necessarily be more revenue-generating this year, but it might be something that next year and the year after is going to be hugely impactful for us. You have to be able to have those conversations.

[00:23:23] When you're having those conversations, having a common language for value is incredibly important, particularly the larger the enterprise is, and your ally in this is the finance team. For people who maybe aren't in a large organization that seems a bit strange, but finance essentially becomes your United Nations. They're the team that will define what the value to King will be. They are responsible for financial reporting, for reporting in a very structured way. So when we make a projection that intervention X is going to have dollar value Y, it's finance who say, "Yes, we believe you," and that cuts through a whole ton of arguments. It also cuts through some of the hacking that happens in large organizations, where people project huge value just so they get the attention of a central capability team. This just nullifies all that and makes everything very clear.

[00:24:15] Aligning the middle is hard. You know what, it's actually fairly straightforward to get the executives in your organization to think this is exciting: great, AI is the next big thing, definitely that's going to be in our strategy, in every bullet point in every report we give, AI is important for us, fantastic. When you talk at the individual contributor level, to the engineers who are doing this work, they're completely bought in. The hard yards are in the middle, the middle tier of, let's say, management, product managers, product owners, where you're trying to essentially disrupt and compete with the alignments and the incentive structures that they have to deliver on day to day. Let me tell you, that's super hard.

[00:24:55] And finally, using ML and AI to do things like make great predictions and lifetime value predictions is great, but that's only the first step. Really, what you do with that, the treatment, and how you deal with balancing your business and optimizing that nonlinear dynamic system using these new tools, is still very hard. I'm going to call it a day there. Thank you very much.

Q&A

[00:25:18] Host: Thank you, Stephen. Stephen, you talked about the messy middle there, but just a question from myself: what was the most unexpected challenge your team faced when putting all this together?

[00:25:31] Steve: The most unexpected challenge. I'm saying something very general and you're asking something very specific. I would say that when you talk to teams and you're trying to talk about the art of the possible, that idea of teams hacking the system is really difficult. I might talk to a team who are responsible for a certain part of our business, and they'll say, "If we can get these people to generate AI solutions, we think we can generate $200 million of additional revenue next year." And the chances are maybe they could, all right. But with finance in place, and with them then having to put that into their projection that they are then bonused on at the end of the year, it cuts through that. So I was surprised at the level of excitement people have, but also the level of over-optimism they have, and the degree to which they're prepared to say things are going to be worth way more just so they can get the attention on their team.

[00:26:25] Host: Yeah, it's a really interesting one. Okay, we'll go to the questions from the audience. What does your product team do to combat addiction? How do you balance this while also optimizing for profit? The resemblance to slot machine design is hard to ignore.

[00:26:39] Steve: Great question. For us, the fundamental premise is that the game is free to play. We absolutely make sure that the experience for people who aren't paying anything and aren't watching ads is still a great experience, and we don't have a business if players are not feeling fully engaged and having a lot of fun. We have lots of systems in place to ensure that's the case. We've also got this amazing community of players. Millions of our players are in this community, sharing ideas and self-helping as well. And if we ever see that a player is having trouble, we take action.

[00:27:08] Host: Okay. Do you believe that AI can replicate the human ability when it comes to finding the fun, which is very nebulous but core to game dev?

[00:27:18] Steve: Very tough. I don't know, and we're still experimenting with that. Finding fun, in a sense, for us is finding that balance of level difficulty, which is at the core of our business. That's why we're investing in creating different AI players who play the levels in advance, so they allow us to test before we release to the audience. And we still have the idea that we have a single progression for all of our players. Everyone experiences the same Candy. It's not as if we optimize level 500 for different types of players. We don't. You get one game, one set of levels. But we're trying to figure out the right journey, the right ramp, the roller coaster ride effectively, so that people get that fun, get that sense of achievement, and then maybe that sense of not achieving, so that it's not all a high.

[00:28:13] Finding fun and defining fun is really, really hard for us. We do actually talk to our players, believe me, and we try to understand what they're finding fun, and then we look for correlates on our quantitative side, looking for those intersections of qualitative and quantitative. It's not easy, but I think we certainly have a lot of signals that we can use to say players are having fun in this area, and then we use that to test and to optimize as we go forward.

[00:28:39] Host: Okay. And do employees feel threatened by the increased productivity that AI represents?

[00:28:47] Steve: Isn't that interesting? The entire industry is thinking about that. Everybody who's like me, let's call it generally a knowledge worker, is thinking, what does this mean? I would say everybody should be absolutely engaging with how these technologies are going to help them in their role, and learning how that role is going to change. It is going to change. I don't think we have a degree of anxiety about this in our organization. I'd say it's more excitement. We've been running internal pilots with hundreds of our King teams in somewhat controlled environments, where we look at what a finance team does when they have a capability like GPT-4 at their disposal, or what software engineers do. And I'll tell you, everybody who gets engaged in those pilots, when we survey them afterwards and ask whether they want to continue to learn and engage with this tool, we have 100% unanimity that they do. I think that's the challenge for us. Everybody just needs to think about what this means for them and their craft, and expect that there's a high likelihood, if you're working in an industry like the software industry, that your role is going to change over the next couple of years as these technologies come to bear.

[00:29:53] Host: Absolutely. Do you test every new level with users? Is game user testing different from standard testing?

[00:29:59] Steve: Well, yes, we test every new level with users, because we release to those users and then we watch like a hawk whether those levels are working. We have A/B testing infrastructure as well that we rely very heavily on. Even though we might run through with AI players first to test for a level of quality, at the end of the day, when we're releasing levels, we're also looking for the signals from the players. We're constantly looking at that and determining: is this more fun, is this less fun, is it causing too much frustration, is it too easy? We use the data from the A/B tests to decide whether we need to make a fix and go back to the drawing board. So yes, it's a combination of both.

[00:30:36] Host: Okay. Is there a risk of AI making young users spend more and more money on microtransactions when they pay to win, and it then becomes a source of gambling [?]?

[00:30:46] Steve: We don't have a pay-to-win mechanic at all. There's no situation where you're winning and someone is losing on the basis of you paying. What we have is people who pay in order to progress a little bit faster, and some people just like doing that, and that's fine. I'm personally someone who doesn't; I just like trying to solve the puzzle. So we're trying to fine-tune our game for all those different sorts of players. And our games are available for people in the 18-plus category. We're targeting adults, we're not targeting children at all, and in jurisdictions where that's the case they're prohibited from playing the game, and we certainly support that.

[00:31:25] Host: Okay. And what's stopping you from letting AI design all your levels?

[00:31:29] Steve: I think it's a long way off, AI doing that. We've got really incredible level designers who are just so great at their craft. What we see these techniques doing is allowing them to be more productive. We'd love to be generating more levels, we'd love to be generating more surprising and sophisticated levels, and we're investing really heavily in the tools that we have for level design, really enabling those level designers to do more with the time they have available to them, and they love this. But think about it: when this is realized, you have a level designer making a choice to change the color of a single block in a tile, in the canvas of the level in the puzzle, and the AI system is saying that might make it 10% harder. That's the sort of interaction we have, and the AI is doing that on the basis of a number of different types of players. We still need our level designers to use their creativity to design a level in a specific way, and I think we're a long way off anything being more automated than that.

[00:32:24] Host: Okay, and we'll go to the final question now. In mobile, speed, loading and all that matter. Is there something with regard to app sizes and stability? Is AI actually going to start clogging things up?

[00:32:35] Steve: It's a great question. The size of the download, and the time it takes to load your game and get to the point where you're playing, correlate directly with retention in your applications, running at the scale that King operates at. So we obsess about this stuff. This is not an issue for us right now. We don't have AI embedded in our client. We're using AI as part of how we produce our game, how we test our content, how we do software development, how we assess the data that's coming back into our systems and generate insights from that. So there's no question of us putting a huge ChatGPT model into the game client. That's not what we're doing at all.

[00:33:13] Host: Ladies and gentlemen, put your hands together for Stephen Collins.