Launching A Cultural Phenomenon: The Next Phase For Niantic Post Pokémon Go

Phil Keslin at UXDX Europe. Video: https://youtu.be/hnuz3VdY6v0

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Where we came from: Earth Viewer and Google Earth

[00:00:03] Hello. I'm here to talk about a company that we built. Before I dive into that, I wanted to talk a little bit about where we came from, why we're doing what we're doing, the things that we've done to date including Pokémon Go, and where we see ourselves going in the future.

[00:00:29] It took a while to find this. This is from 2001, which was when we launched the first version of Earth Viewer, which was the predecessor to what is today Google Earth. John and I started this company with a few others in March of 2000, and we went on for several years until the company was acquired by Google. The dream for the original Earth Viewer was to collect as much information about the earth, present it in a very interactive form, and allow people to share information about the world in this geo-specific context, and stream that over an internet that could go no faster than 256 kilobits per second, with nothing better than a Pentium processor.

[00:01:19] Over time technology has changed. The Google acquisition allowed the organization to acquire a mass amount of data, which has really enabled Google Earth and Google Maps to become pretty much the preeminent map source on the planet. Street View was one of the pieces that the company funded. You can imagine how much money they spent to send cars all over the world, drive around the streets and collect imagery, so they could not just capture the streets but understand where all the addresses are, where all the special features are. We even built special trucks like this bicycle, and the little cart. The cart is actually used in museums and gives you interesting walkthroughs through museums. Again, it's all about sharing information about the world.

Getting people off screens and outside

[00:02:06] So in about 2010 John was getting kind of antsy. He's the CEO of the company, and he's trying to figure out what he wanted to work on next, and what he observed was that a lot of people were spending an enormous amount of time in front of screens. You can see here the child playing Minecraft. This cover of The New Yorker is pretty poignant: it shows two girls sitting in their bedroom playing Minecraft in an outdoor environment, while sitting outside the window is a beautiful playground. And this is actually pretty interesting, because my daughters used to do this. They used to stay in each of their own rooms with their doors closed, with their laptops on their laps while they laid in their bed, and played Minecraft outside.

[00:02:48] So what were we going to do about it? We decided that our objective was to find ways of motivating people to get outside and explore the world around them, maybe get a little exercise along the way, and experience other people through real world social experiences. In many of these cases, discovering the world around you, there's some really interesting stuff around the world. And for exercise, just getting people out walking around, that would be sufficient for what we wanted to do.

[00:03:32] But in order to get as many people motivated as possible, we need to create as many applications as possible, and really, in order to do that at scale, you have to create a platform. And so our original goal was to create that platform, but we didn't know what that platform should look like. So instead of building the platform from the get-go, we started with applications, and we picked two.

Field Trip and the first step into AR

[00:03:53] The first one was Field Trip. Most people don't know much about Field Trip, it was a pretty small distribution. Field Trip is a content publishing and distribution system using location as the mechanism for triggering notifications about that location. Most of the information that went into it is heavily curated. Organizations like Thrillist provided content, a small company called Arcadia Publishing, which builds these little 44 page pamphlets about neighborhoods using local experts. Those were the initial feeds.

[00:04:29] In fact, it was the name of our company, Niantic, that we spawned. That was the genesis of this particular project. John was walking through San Francisco, walked by the Transamerica Tower, and on it is a plaque describing the location as the last resting place of the whaling ship the Niantic. It was a ship that was abandoned in the bay, pulled on shore, turned into a hotel, I think a brothel, a bar. Nobody really knows exactly when they went in and did the landfill in the financial district, but they left a lot of these ships underneath those buildings, which they're still under.

[00:05:06] So from that genesis, that epiphany idea, we started to play around with how can we deliver this content to people in really interesting ways, and it was the first application of augmented reality that we actually explored. You'll see on the right-hand side of that image an image that was presented on Google Glass, so the viewfinder is the little snippet in the upper right-hand corner. So, first step into AR. Then we created a game called Ingress.

[00:05:48] Video clip: What was the net effect of the Niantic project? We had crossed a threshold in which global security could be at risk. Decrypting the data was the mistake. This is not psychosis or some cognitive break, but an actual takeover of the mind. Much of the public sculpture found in our cities is based on design seeded in the human mind. Certain places have an energy that not only attracts people but attracts events. The mission of 13MAGNUS is to monitor the effects of mind hacking. Obviously this will be done with the highest of security, to make sure that the ideas do not contaminate or threaten humanity. This all leads to Niantic tech[?]. I know that many tools will be needed to fight this battle. You just have to know where to look and know what you're seeing. Portals emit exotic matter into our world, and that matter has certain effects on our world. I started noticing that there were energy fields, anomalies on earth all around me. A few of them exhibit properties that are as yet unexplained. I know that there are others out there. What if they're already among us but we don't realize it? And I must be prepared to work with them or fight them. They are coming. Something's wrong out there in the world. This doesn't feel like a scientific study. The one hope lies in understanding what happens at Niantic. Not all mysteries are solvable, but the joy comes in the pursuit.

What Ingress was an exploration in

[00:07:23] So the basic premise of the game is aliens from another dimension have entered our world through these high-energy locations that we called portals. There are individuals who are sensitive to the energy field, namely artists, and they go off and create these great artworks and they put them in these locations where they've experienced this energy field. And we use that particular backdrop to collect all this data about interesting places on the planet. We started with the seed of public artwork, which is actually contributed by users through the Panoramio database, and then we added a feature into Ingress to actually collect more of that, and today there's about six and a half million of them that are actually populating the world database.

[00:08:18] So Ingress was an exploration in how do you make location-based games that are engaging, fun, and get people to sort of help us with our mission. We didn't really think about how do you make money doing this. We were inside Google, so monetization wasn't really a huge thing, it was sort of getting users up. So the thing we decided to explore was location-based sponsorships.

[00:08:46] Here are four of the early sponsors for the game. I don't know if any of you are familiar with Hint Water, for the United States. They were one of our first launch partners. Inside every cap, for the first couple of years, there was an eight character code under there that if you entered it into Ingress would give you special goodies. AXA was another partner, which I think most people here would be familiar with, Zipcar in the US, and Lawson in Japan. The goal there was to find ways to drive traffic into these locations, use it as a monetization vehicle. And that's where we started. We played around with that for a while, it was actually pretty successful, and so we built infrastructure to help us with that.

[00:09:34] The last piece I'm going to talk about is events. This is the first event that we did, in Cahokia Mounds in January of 2013, about two months after we launched. There was a handful of people, it was cold and rainy and they still showed up. They drove in from several states to actually play for about three or four hours for this particular event, and we thought it was great. So that spawned our series of events. We started doing them every month, doing sponsored community days. We eventually evolved that into these arcs that were three months long. Each month there would be one large event with some smaller events alongside of it, and it would culminate in a large event at the end of the cycle. And in July of 2016 we had the largest event, in Tokyo, which had 10,000 people show up for it at least, and this is a picture of all those people.

[00:10:32] So with Ingress, we set out to build an engaging real-world game. We tried to learn how to leverage players and our partners to create that game board. We wanted to make location-based sponsorship work. We wanted to foster a sense of community to drive that engagement. And from that we designed a new platform whose goal was to increase efficiency and reduce our operating cost by 10x, that could handle something at scale across the planet.

[00:11:08] Video clip: Amazing creatures have been discovered. Now more than ever, undiscovered species appear. Oh, come on.

Building Pokémon Go

[00:11:27] In 2014 we partnered with the Pokémon Company, the beginning of construction of Pokémon Go. This was born out of an idea. I get this question a lot, it's like, how'd you guys come up with this idea? It's actually born from an April Fool's joke. On April 1st 2014 there was a "collect all the Pokémon on Google Maps," and we saw that that was a pretty big hit. I know my wife did nothing but collect Pokémon roaming around Google Maps while it was live. And so we decided that this kind of made sense, and we talked to the Pokémon Company about it and to them it made a lot of sense, and that helped us develop this partnership for building this game.

[00:12:08] We started building it at the time with a team of about eight people, eight to ten people working on the product inside of Google. They were building both the platform and the game in tandem. We spun out of Google in October of 2015 and continued to work on the game. That was a pretty disruptive process, took about six months to do that. We're now an independent company, and then nine months later we launched Pokémon Go.

[00:12:42] We launched the game on July 6th of 2016, and this is what happened. So everybody's like, oh, did you expect that? It's like, no, we did not. This is a group of people in Taipei running to catch Snorlax. There have been similar events going after Lapras in Central Park. People were stopped in the middle of the street trying to pick up these Pokémon, they'd just get out of the car and run into a park and do these things. So this is not the norm, this is just the extreme cases, and the magic of YouTube sort of allows us to show these things in all their glory.

[00:13:20] So this is kind of what happened. You can imagine what it looked like to us on the inside. This is the graph of the performance of the system at launch. So if you can read it, the orange line, for those of you who aren't colorblind, orange is where we thought we were going to be. The red line was our worst case, and we actually used real numbers. We didn't just make something up. We looked at the top MMOs and we took their peak utilization and we multiplied that number by five, and we thought that would be really good. If we could make sure we had a system that could support that amount of traffic, that would be awesome. One thing you'll notice is the green line doesn't even start at the orange line. That was New Zealand and Australia. The USA followed on the next day, and it was like a rocket ship after that.

[00:14:15] So needless to say we ran into a lot of bottlenecks. Anytime everybody says we need to do a 10x scale, things get hard, and when you get to half of that it's even harder. The most interesting thing was that this is the server team, and what's interesting about this is, notice there are four people in that photo. That's how many server engineers were supporting that. So needless to say, for the first six weeks after launch life was pretty crazy. People were sleeping on sofas, on floors.

[00:14:47] But the thing that really saved that group of team members was, one, we had built a platform that was optimized to reduce cost, so it was a high-performance system, so it could actually handle this load. So we actually designed this in from the beginning. We didn't think it was going to get this big that fast, so we never tested it at that scale, but we actually designed it to handle this kind of a load. And the best part was that, after you've been struggling trying to get these machines up that had been down for three and a half hours and people are yelling at you, screaming, you can go outside after you get everything up and stable again and you can see the smiles on people's faces as they're playing your game. And they were everywhere. That was awesome.

Sponsorships, lures and small businesses

[00:15:32] Sponsorships. We did pursue sponsorships. I don't know how many of you here play Pokémon Go, but Starbucks, at least in the US, are sponsored locations. We have several other ones. We've integrated sponsorship into the platform now, with tools for ingesting places, their locations or store locations, and analytics, so they actually get feedback on whether people actually come to their store.

[00:16:00] One thing that we discovered after we launched Pokémon Go was that there's a feature called the lure, which you can attach to a PokéStop and it causes Pokémon to spawn. What we discovered was that small businesses were using that to place it on the nearby PokéStop, and then they put a placard up in front of their store saying if you play Pokémon, come in and show us your game and we'll give you a discount on products. And so they basically created their own self-service sponsorship. They paid for the lure, and as far as cost of the lure versus what they got, they could go out every half hour and put another lure on it and they could actually generate a pretty fair amount of revenue. There was actually an ice cream store just outside of Seattle, I think, or Oregon, that tripled their revenue and it actually saved the store.

Events: Go Fest, Yokohama and community partners

[00:16:50] Then we started doing events. In July of 2016 we did our first event, it was Go Fest in Chicago. Ahead of that we had added the raids feature, which would allow us to get the legendary Pokémon into the game, and the raids feature required that multiple players actually engage in combat in order to defeat this legendary Pokémon. And so we thought we had all the ingredients in place to make Go Fest a reality. What we underestimated was the impact of 20,000 people in a single location with limited cellular network capacity, and it obliterated us. It was a true lesson learned.

[00:17:33] The thing that the team did, though, in order to improve that situation, is they distributed the Pokémon that would have only occurred in the park throughout the city, and they dispersed the crowd over the course of the remainder of that day. And actually what people saw as a disaster turned into a success, because people still had a lot of fun throughout that particular activity, and people traveled from all over the place in order to participate, from other countries in fact.

[00:18:01] We followed Go Fest with Pikachu Outbreak a month later. Pikachu Outbreak is a festival that the Pokémon Company puts together, and it's an opportunity for them to get all these people to show up in Yokohama. And we blew it up, it actually worked incredibly well. That's when we launched Mewtwo, we did our first stadium event. We followed Yokohama with the Safari Zone in Europe and in the malls.

[00:18:27] One thing we've done with events, however, is we've started to partner with community organizations to foster getting people outside and into their community. These are three of the organizations that we partnered with over the last year. CicLAvia, which is in Los Angeles: they close their streets down for basically skating, walking, playing out in the streets, and we partnered with them, brought a lot of people into it. Philly Free Streets does a similar activity, they ended up having a really successful event. And the Big Heritage foundation in Chester, United Kingdom, is an organization that wants to teach people about Chester and the heritage in that community. They did a weekend with us in March of last year, and over the course of that weekend it attracted 16,000 people, which is a pretty enormous turnout. And in the summer we did Earth Day in partnership with Mission Blue. We ended up having, I think, 4,200 people, and six metric tons of trash was collected in 68 cleanup events in nine different countries.

[00:19:44] We continue from there. This last summer was our blitz, I think every weekend, which is punishing for the engineering team. We put on some major event, one really large one, every month in June, July and August. The first one we did was in Dortmund, and its Safari Zone.

[00:20:09] Video clip: I've already made a few friends, met people from all over the world. I've heard something like 20 different languages. It's been great so far. I was too excited about the event that I... yesterday. I've been playing since it first came out, so I started the game when it was introduced in Germany, it's two years ago. And when Niantic released Pokémon Go in July 2016, everyone wanted to relive their childhood. They wanted to relive the dream of going out in the world, exploring, finding Pokémon out in the wild. My goal actually, coming here to the event, is expanding my friends, not just local friends in my neighborhood but also to make international friends. I ended up catching a Pokémon in every single country I went to, and it's something special to keep. You take it home and you're like, look, I went to that country and I have something like a little keepsake you take home with you.

[00:21:26] A two-day event attracted over 50,000 people just in the park, and close to 200,000 people, I think, in the surrounding community, coming from all over the place just to participate in that game. We did a similar thing in Chicago, which is the place of our original. Going back there we felt was pretty brave, but we decided we owe it to the community to do that. And last year it was a huge success. I actually got to go this year, and just seeing the smiles on people's faces was just phenomenal.

What we got from Pokémon Go

[00:22:02] So what did we get from Pokémon Go? We were able to leverage key learnings from Ingress. Everybody talked about how Pokémon Go is in its demise, and it's actually still in one of the top ten spots. This summer's push has actually helped quite a bit, but it's not in decline, the actual game is growing. It is over two years old. It's the first product that we launched on our new platform, we're since rebuilding Ingress on top of it, and while it was a wild ride, it proved that the architecture that we built would scale and that the infrastructure is capable of supporting where we want to go in the future.

[00:22:44] At our company, everybody talks about KPIs, everybody talks about revenue and monthly active users and daily active users. At our company, because exercise is one of the key components, we talk about how many kilometers our players walk. And we feel pretty confident to say that for one of our titles players have walked more than the combination of all our other titles combined, and that's 340 million kilometers, and that's Ingress. Pokémon Go is 22 billion kilometers, which is the distance the earth travels around the sun in over 22 years. So it's a lot of miles. And then we're going to come up with Harry Potter. I can't really talk about that since we're not ready to talk about it, but look for that next year.

[00:23:39] So this is our architecture. You can see it's got a lot of pieces. Every single one of the boxes inside that big horn[?] square represents the level of experience and knowledge that we have poured into the core infrastructure.

Where we're going next: shared AR experiences

[00:23:57] So where are we going next? AR is a big thing for us. We believe that in order to truly enable social experiences, real world social interactions, you have to have shared social experiences. In every single case, when you hear people talking about it, it's about the interactions of those people. And so last year we acquired a company out of the UK that does this.

[00:24:24] This is a machine learning model that uses nothing more than a camera to place Pikachu and his friend into the scene and determine the depth of the other objects in the scene. So while this is a video and a pre-rendered Pikachu, the occlusions that are occurring are happening in the software. And actually our goal is to get that running on cell phones, so that when Pikachu runs around he actually goes in and out of things that are in motion in the scene.

[00:24:56] And with the platform we built a new experimental game. It took us about two weeks to get it, and this is what we ended up with.

[00:25:13] So the game operates, there's little pellets on the ground. The goal was to force people to move, so you have to run around to actually collect the energy that you need in order to shoot the energy weapon, if you call it that. And so that's the reason why people are in motion. What you're seeing is the individuals with an over-the-shoulder view, plus there's a separate iPad that's actually in the game but not actually participating, that's able to observe all of this. So they built this as a prototype for a press event that we did in June. It took them about two weeks to get to this point, after a lot of iteration around the things.

[00:25:51] So this is what we hope to achieve with our product in the future. Given the capabilities of the occlusion systems, and there's some other capabilities we're adding to it as well, we think that we can build some really compelling content people can go out and play and have a great time together. That's it for me.