Skip to main content

Gran Turismo Sophy devs explain how it taught an AI racer sportsmanlike conduct

As the first significant public release of technology from Sony AI, which was formed in 2020, the upcoming Gran Turismo Sophy Race Together mode for Gran Turismo 7 will be many players’ first exposure to complex AI technology that could have a lasting impact on many of Sony Interactive Entertainment’s games. As such, if the AI is rude, unfair, and unbeatable for many players, GT Sophy could be seen as yet another warning sign that AI isn’t ready for primetime yet. That’s why ensuring GT Sophy has proper sports etiquette is a priority for its developers.

In racing, you need to make sure that you’re driving aggressively because you do want to win, but at the same time, you want to drive fairly,” Sony AI COO Michael Spranger tells Digital Trends. “You can’t bump into other cars in order to gain an advantage and need to make sure that everybody can fairly compete. That’s an interesting area because these rules are quite imprecise; there’s a rulebook, but it needs interpretation from judges. Getting this right — driving aggressively and competitively versus driving fairly — is one of the big challenges of GT Sophy.”

Sony AI, Sony Interactive Entertainment, and Polyphony Digital have confirmed that players will be able to race GT Sophy in a limited-time Gran Turismo 7 mode starting on February 21. To get to that point, its developers did a lot of work refining not only GT Sophy’s skill via reinforcement learning but its etiquette toward a nebulous set of courtesy-driven rules as well. Ahead of its addition to Gran Turismo 7, Digital Trends spoke to Sony AI COO Michael Spranger and GT Sophy Project Lead Peter Wurman to learn more about their ethical approach to GT Sophy’s sportsmanlike conduct, and what future uses they see for this technology in gaming.

The following interview has been edited for brevity and clarity. 

What made Gran Turismo the most appealing option to develop an advanced AI for first?

A headshot of Sony AI COO Michael Spranger, PhD.
Michael Spranger, PhD, COO of Sony AI

Michael Spranger: It was because it was possible and because Gran Turismo is a really exciting game. Gran Turismo has been part of PlayStation from the very beginning. It’s an iconic game that provides this specific challenge area of physical realism, strategy, decision-making, and sports etiquette. Also, it has a big audience of people actually following the game, and that means that any technology we develop in the game can have a potentially meaningful impact on millions of players. I also love the game. These things combined made us choose Gran Turismo as one of the first targets for gaming AI projects.

How does reinforcement learning allow this AI to take more risks or drive more impressively than your standard game AI? Is it just trying to drive the most precise racing line? 

Peter Wurman: There are different levels of skill. The baseline is driving faster on the track, so reinforcement learning works by letting the AI agent explore what effect its actions have on the world and then try to maximize its reward. By giving it positive signals for going faster on the track, it learns to drive around the track very fast. It learns to press on the accelerator and break into corners so it doesn’t crash, which works great for training it for a time-trial scenario.

To actually make it a competitive racer, we had to give it other reward signals for passing cars that are in front of it and not preventing cars behind it from passing it. We also gave it penalties for colliding or bumping into other cars. This combination of rewards and penalties took some effort to get right, but once we did training with the right population of opponents, it learned to be a very effective racer.

A headshot of GT Sophy Project Lead and Director of Sony AI America Peter Wurman.
Peter Wurman, PhD, Director of Sony AI America and Project Lead on GT Sophy

What were the biggest learnings from the first couple of Race Together events that you held with top Gran Turismo Sport players?

Peter Wurman: If you’ve followed along with the project, you’ll know there were two races. The first we did well in, but we didn’t win the Team Score. Some of the interesting things we learned from that had to do with finding the balance of assertiveness versus being a good sport. We found out two weeks beforehand that the agent we had trained was too aggressive and wouldn’t be allowed to race, so we had to scramble and adjust things. You can’t have one AI agent that was really good at driving fast and another that thought about the tactical aspects of passing; instead, we had to incorporate them together into one coherent agent that was doing both at the same time. Those are the things we saw after the July 2021 race, so we felt much more confident going into the October 2021 race.

Michael Spranger: There was this thing where we were told it was too aggressive, but then on the first exhibition race day, we actually lost the race because were not aggressive enough and timid. The problem of how you balance those things was really apparent across the development of the project.

A replay of a Gran Turismo Sophy Race Together match in Gran Turismo 7.

What changes had to be made to GT Sophy when you made the decision to add it to Gran Turismo 7 and not just train it in a test environment? 

Peter Wurman: The biggest is that in all the exhibitions, GT Sophy ran on a separate computer and talked through a networking connection to the PlayStation console. Here, GT Sophy is embedded in the game. Making it work on a PlayStation with the resources that were available to us after all of the rendering and other stuff that’s going on was a bit of engineering work.

The other thing is that Polyphony Digital really wanted the agent to be an extremely good example of racing style and racing skill. We put in some changes to encourage GT Sophy to stay within the lines more. If you’ve watched any of the exhibitions, you’ll see that it was really good at using all of the track, sometimes putting two tires in the grass. That’s legal, but not exactly what you could do in a real race car. We made some changes to keep GT Sophy between the lines more. It’s still very fast but a little cleaner.

We’re mostly focused on trying to adhere to the rules of sportsmanship in automotive racing.

What are the main things the Sony AI team is hoping to learn from adding Gran Turismo Sophy Race Together to Gran Turismo 7?

Peter Wurman: The main one is that we hope people enjoy playing against it, feel like they’re really challenged at all levels of skill, and are learning to become better-simulated racecar drivers.

Michael Spranger: We also want to see how people interact with the technology. Right now, it’s a handful of people who have experienced the technology, so we’re curious to see how people are going to use this to learn for themselves, gain new skills, and have new experiences. I think that’s at the core of the project in many ways. We have the technology breakthrough, and I think that’s really important, but I think we’re unlocking a brand new stage for people and in Gran Turismo. In many ways, the built-in AI has a quite narrow band of performance, and you can surpass that once you become an intermediate driver on each difficulty.

Even with the relatively few track-car combinations that we unveiled here, from what I can do to what some of the top players can do, everybody can find a really interesting match. We hope to see more of that and see how people learn from that. It’s also very difficult for some players to find good opponents, and for the first time, you can have repeated interactions with a worthy opponent without having to call up all your friends and have them all come together. It’s going to be something where people can make a lot of progress in getting better at the game because they can have these tailored experiences to some extent.

A first-person perspective of a Gran Turismo Sophy Race Together race in Gran Turismo 7.

On the ethical side of things, can you explain the process of determining what constitutes sportsmanlike conduct for GT Sophy so it isn’t harassing certain players?

Peter Wurman: Well, it can’t talk to you, so there are a lot of bad things GT Sophy can’t do because that’s not part of the game. We’re mostly focused on trying to adhere to the rules of sportsmanship in automotive racing; now, those are pretty nebulous. You can’t block a legitimate passing opportunity and things like that. There are a bunch of fuzzy words in these rules that were hard to encode and program, so we had to approximate it with reward signals and penalties if it did things like bump another car.

Then, we had to give it to Polyphony Digital and test drivers and let them give us feedback because racing against other AI agents doesn’t really show us if it’s ready to be racing against the variety of human racing styles. That took some effort and iteration to get that right.

Michael Spranger: For this release, I think it’s fair to say that it is competitive. It is going to try and overtake players, but it’s not going to divebomb them. It’s competitive but polite. It’s definitely putting pressure on you; you’re going to see it in the rearview mirrors trying to overtake you. At the same time, it’s going to try and give you some space so you can race competitively but fairly.

Peter Wurman: But if you make a mess, it’ll take advantage.

Michael Spranger: That’s right, instantly!

Gran Tursimo Sophy Race Together mode's race option menu in Gran Turismo 7.

Where do you see GT Sophy and AI technology in Sony games going over the next five years?

Michael Spranger: Obviously, we can’t announce anything specifically, but in general, there are multiple layers to it. There are still challenges and things we can do within Gran Turismo to unlock new experiences, and I don’t think we’re finished with that. There’s real excitement about pushing the technology further. Then, from the Sony AI perspective of corporate R&D and strategic initiatives, we’re really excited about what this technology can bring to gaming in general.

Other games will give us a different challenge to overcome and will further the capabilities of the technology. It’s an emerging technology that has had some successes in the past, but I do think what we’re doing here is quite distinct in the sense that we do the scientific breakthroughs but then we also deploy it and make sure that people get their hands on the AI technology. That is going to push the technology forward and unlock greater experiences.

Interaction is at the core of PlayStation, it’s not an error that it’s called Sony Interactive Entertainment. It’s really about what your interaction with the things that you find in the game is, and I do feel that this technology has the potential to really unlock the next generation of games, and we’re really excited about that.

Editors' Recommendations

Gran Turismo 7 is back online following update error
Three cars come racing around a tight turn in Gran Turismo 7.

After going offline, leaving most of its features unplayable for any users for over 24 hours, Gran Turismo 7 is back up, with its developer, Polyphony Digital, explaining just what happened.

In a post on the game's website, Polyphony Digital president Kazunori Yamauchi explained that what should have been a simple update rollout led to its latest PlayStation exclusive racing title being unavailable for a day. "Immediately before the release of the 1.07 update," reads the post, "we discovered an issue where the game would not start properly in some cases on product versions for the PS4 and PS5." The developer went on to explain that this error could have somehow impacted users' save data, leading to the game's outage.

Read more
PlayStation VR2 is lighter and larger than PS VR
The PlayStation VR2 headset along with its controllers.

In a post on the PlayStation Blog, Sony revealed the final design for its upcoming PlayStation VR2 headset, which it claims will be lighter than the original PS VR headset, all while being larger and packing more features.

Read more
OlliOlli World devs share their perfect skate park recipe
The OlliOlli World player character grins will grinding down a rail in the OlliOlli World cinematic trailer.

OlliOlli World delights as a side-scrolling skateboarding game where players travel through the land of Radlandia in order to become a skate wizard and enter the mystical land of Gnarvana. It puts players into a flow state where they tune out of everything else and focus solely on its mesmerizing gameplay. As the game pulls this off so well, it's not surprising that developers got caught up in that feeling when designing OlliOlli World's masterful levels.

OlliOlli World – Official E3 2021 Trailer

Read more