Niantic — the company behind Pokémon Go and the upcoming Harry Potter: Wizards Unite — is on a mission to make augmented reality more realistic. To truly augment reality, computer systems need to be able to understand the real world a little more, and Niantic thinks the technology isn’t quite there yet.
That’s why it’s launching the Niantic Real World Platform, a program that aims to tackle today’s issues with AR on multiple levels. The platform’s launch comes as the company also announces the acquisition of Matrix Mill, a U.K.-based AR research and development firm developing neural networks that can infer information about the world using data from your smartphone’s camera.
The Niantic Real World Platform is first going to attempt to “model reality.” Niantic wants the platform to be able to adapt and change as the real world changes. In the real world, buildings are constructed, parks are formed, roads are closed, and cities expand — if there’s a significant delay in the platform adapting to those changes, it can affect game-play. The digital model of the world that Niantic has needs to be able to adapt almost in real-time, and it requires a combination of computer vision and machine learning.
Niantic also said its platform needs to be able to “understand reality,” meaning it needs to be able to see an object, recognize exactly what it is, and what the object’s place is in a broader environment. Computer vision is key here — Niantic is training its platform to accurately recognize objects based on data from a phone’s camera, and as time goes on, more and more objects will be added to the platforms “AR vocabulary.”
Recognizing objects is only one part of the puzzle, as systems need to learn where those objects are in a 3D space. Simply knowing that an object is a car doesn’t necessarily matter in the context of a game if that car is 50 feet away. That’s why Niantic is working on developing better depth maps, which can understand exactly how far away an object is at any given time. That can help digital elements better interact with the real world.
The platform will also try to cut down latency, which is when you experience delays in multiplayer games thanks to connectivity issues. Latency in multiplayer games can also affect game-play, and Niantic showed off a demo game called “Codename: Neon,” which involved people moving around an environment, collecting balls of light, and launching them at other players to get points. The interactions were smooth and seamless between all the players, and it was a testament to the progress Niantic has made to reduce latency in its platforms.
Niantic is throwing a lot of resources into its AR tech, and it’s work that it won’t be keeping all to itself. The Real World Platform will be available to third-party developers in the future, and while more information will be available in the coming weeks, Niantic said it’s selecting a few developers to work with later this year. Hopefully, this means we’ll start seeing better-looking and better-performing AR games in the future.
Niantic’s CEO John Hanke said games like Pokémon Go “never have to end.” Hanke envisions continuously improving games that are such a huge part of people’s lives, that they pass them on to their children.
