Skip to main content

Gran Turismo Sophy devs explain how it taught an AI racer sportsmanlike conduct

As the first significant public release of technology from Sony AI, which was formed in 2020, the upcoming Gran Turismo Sophy Race Together mode for Gran Turismo 7 will be many players’ first exposure to complex AI technology that could have a lasting impact on many of Sony Interactive Entertainment’s games. As such, if the AI is rude, unfair, and unbeatable for many players, GT Sophy could be seen as yet another warning sign that AI isn’t ready for primetime yet. That’s why ensuring GT Sophy has proper sports etiquette is a priority for its developers.

In racing, you need to make sure that you’re driving aggressively because you do want to win, but at the same time, you want to drive fairly,” Sony AI COO Michael Spranger tells Digital Trends. “You can’t bump into other cars in order to gain an advantage and need to make sure that everybody can fairly compete. That’s an interesting area because these rules are quite imprecise; there’s a rulebook, but it needs interpretation from judges. Getting this right — driving aggressively and competitively versus driving fairly — is one of the big challenges of GT Sophy.”

Sony AI, Sony Interactive Entertainment, and Polyphony Digital have confirmed that players will be able to race GT Sophy in a limited-time Gran Turismo 7 mode starting on February 21. To get to that point, its developers did a lot of work refining not only GT Sophy’s skill via reinforcement learning but its etiquette toward a nebulous set of courtesy-driven rules as well. Ahead of its addition to Gran Turismo 7, Digital Trends spoke to Sony AI COO Michael Spranger and GT Sophy Project Lead Peter Wurman to learn more about their ethical approach to GT Sophy’s sportsmanlike conduct, and what future uses they see for this technology in gaming.

Prime Day Focus
These Razer Blade Prime Day deals really pack a punch [in gaming power]
Secretlab Prime Day deals: Build your ideal work-from-home or gaming station
Amazfit Amazon Prime Day deals dropped: Save up to 40% off smartwatches today
Send it! This HoverAir X1 Drone can capture your adventures and it's $120 off

The following interview has been edited for brevity and clarity. 

What made Gran Turismo the most appealing option to develop an advanced AI for first?

A headshot of Sony AI COO Michael Spranger, PhD.
Michael Spranger, PhD, COO of Sony AI Image used with permission by copyright holder

Michael Spranger: It was because it was possible and because Gran Turismo is a really exciting game. Gran Turismo has been part of PlayStation from the very beginning. It’s an iconic game that provides this specific challenge area of physical realism, strategy, decision-making, and sports etiquette. Also, it has a big audience of people actually following the game, and that means that any technology we develop in the game can have a potentially meaningful impact on millions of players. I also love the game. These things combined made us choose Gran Turismo as one of the first targets for gaming AI projects.

How does reinforcement learning allow this AI to take more risks or drive more impressively than your standard game AI? Is it just trying to drive the most precise racing line? 

Peter Wurman: There are different levels of skill. The baseline is driving faster on the track, so reinforcement learning works by letting the AI agent explore what effect its actions have on the world and then try to maximize its reward. By giving it positive signals for going faster on the track, it learns to drive around the track very fast. It learns to press on the accelerator and break into corners so it doesn’t crash, which works great for training it for a time-trial scenario.

To actually make it a competitive racer, we had to give it other reward signals for passing cars that are in front of it and not preventing cars behind it from passing it. We also gave it penalties for colliding or bumping into other cars. This combination of rewards and penalties took some effort to get right, but once we did training with the right population of opponents, it learned to be a very effective racer.

A headshot of GT Sophy Project Lead and Director of Sony AI America Peter Wurman.
Peter Wurman, PhD, Director of Sony AI America and Project Lead on GT Sophy Image used with permission by copyright holder

What were the biggest learnings from the first couple of Race Together events that you held with top Gran Turismo Sport players?

Peter Wurman: If you’ve followed along with the project, you’ll know there were two races. The first we did well in, but we didn’t win the Team Score. Some of the interesting things we learned from that had to do with finding the balance of assertiveness versus being a good sport. We found out two weeks beforehand that the agent we had trained was too aggressive and wouldn’t be allowed to race, so we had to scramble and adjust things. You can’t have one AI agent that was really good at driving fast and another that thought about the tactical aspects of passing; instead, we had to incorporate them together into one coherent agent that was doing both at the same time. Those are the things we saw after the July 2021 race, so we felt much more confident going into the October 2021 race.

Michael Spranger: There was this thing where we were told it was too aggressive, but then on the first exhibition race day, we actually lost the race because were not aggressive enough and timid. The problem of how you balance those things was really apparent across the development of the project.

A replay of a Gran Turismo Sophy Race Together match in Gran Turismo 7.
Image used with permission by copyright holder

What changes had to be made to GT Sophy when you made the decision to add it to Gran Turismo 7 and not just train it in a test environment? 

Peter Wurman: The biggest is that in all the exhibitions, GT Sophy ran on a separate computer and talked through a networking connection to the PlayStation console. Here, GT Sophy is embedded in the game. Making it work on a PlayStation with the resources that were available to us after all of the rendering and other stuff that’s going on was a bit of engineering work.

The other thing is that Polyphony Digital really wanted the agent to be an extremely good example of racing style and racing skill. We put in some changes to encourage GT Sophy to stay within the lines more. If you’ve watched any of the exhibitions, you’ll see that it was really good at using all of the track, sometimes putting two tires in the grass. That’s legal, but not exactly what you could do in a real race car. We made some changes to keep GT Sophy between the lines more. It’s still very fast but a little cleaner.

We’re mostly focused on trying to adhere to the rules of sportsmanship in automotive racing.

What are the main things the Sony AI team is hoping to learn from adding Gran Turismo Sophy Race Together to Gran Turismo 7?

Peter Wurman: The main one is that we hope people enjoy playing against it, feel like they’re really challenged at all levels of skill, and are learning to become better-simulated racecar drivers.

Michael Spranger: We also want to see how people interact with the technology. Right now, it’s a handful of people who have experienced the technology, so we’re curious to see how people are going to use this to learn for themselves, gain new skills, and have new experiences. I think that’s at the core of the project in many ways. We have the technology breakthrough, and I think that’s really important, but I think we’re unlocking a brand new stage for people and in Gran Turismo. In many ways, the built-in AI has a quite narrow band of performance, and you can surpass that once you become an intermediate driver on each difficulty.

Even with the relatively few track-car combinations that we unveiled here, from what I can do to what some of the top players can do, everybody can find a really interesting match. We hope to see more of that and see how people learn from that. It’s also very difficult for some players to find good opponents, and for the first time, you can have repeated interactions with a worthy opponent without having to call up all your friends and have them all come together. It’s going to be something where people can make a lot of progress in getting better at the game because they can have these tailored experiences to some extent.

A first-person perspective of a Gran Turismo Sophy Race Together race in Gran Turismo 7.
Sony Interactive Entertainment

On the ethical side of things, can you explain the process of determining what constitutes sportsmanlike conduct for GT Sophy so it isn’t harassing certain players?

Peter Wurman: Well, it can’t talk to you, so there are a lot of bad things GT Sophy can’t do because that’s not part of the game. We’re mostly focused on trying to adhere to the rules of sportsmanship in automotive racing; now, those are pretty nebulous. You can’t block a legitimate passing opportunity and things like that. There are a bunch of fuzzy words in these rules that were hard to encode and program, so we had to approximate it with reward signals and penalties if it did things like bump another car.

Then, we had to give it to Polyphony Digital and test drivers and let them give us feedback because racing against other AI agents doesn’t really show us if it’s ready to be racing against the variety of human racing styles. That took some effort and iteration to get that right.

Michael Spranger: For this release, I think it’s fair to say that it is competitive. It is going to try and overtake players, but it’s not going to divebomb them. It’s competitive but polite. It’s definitely putting pressure on you; you’re going to see it in the rearview mirrors trying to overtake you. At the same time, it’s going to try and give you some space so you can race competitively but fairly.

Peter Wurman: But if you make a mess, it’ll take advantage.

Michael Spranger: That’s right, instantly!

Gran Tursimo Sophy Race Together mode's race option menu in Gran Turismo 7.
Image used with permission by copyright holder

Where do you see GT Sophy and AI technology in Sony games going over the next five years?

Michael Spranger: Obviously, we can’t announce anything specifically, but in general, there are multiple layers to it. There are still challenges and things we can do within Gran Turismo to unlock new experiences, and I don’t think we’re finished with that. There’s real excitement about pushing the technology further. Then, from the Sony AI perspective of corporate R&D and strategic initiatives, we’re really excited about what this technology can bring to gaming in general.

Other games will give us a different challenge to overcome and will further the capabilities of the technology. It’s an emerging technology that has had some successes in the past, but I do think what we’re doing here is quite distinct in the sense that we do the scientific breakthroughs but then we also deploy it and make sure that people get their hands on the AI technology. That is going to push the technology forward and unlock greater experiences.

Interaction is at the core of PlayStation, it’s not an error that it’s called Sony Interactive Entertainment. It’s really about what your interaction with the things that you find in the game is, and I do feel that this technology has the potential to really unlock the next generation of games, and we’re really excited about that.

Tomas Franzese
Tomas Franzese is a Staff Writer at Digital Trends, where he reports on and reviews the latest releases and exciting…
This is the perfect moment to buy a Meta Quest 3 for gaming
The Quest 3 and Touch Plus controllers appear in Meta's charging dock accessory.

A person is enthusiastically enjoying a game while wearing the Meta Quest 3. Meta

At this point, you're probably bought in to VR gaming or you've not. The tech has a notoriously mixed reputation among gamers, some of whom have embraced VR as a gaming option and others who still reflexive sneer at it. I've been a supporter ever since the original PlayStation VR launched as I've played plenty of great games using the four headsets I've owned over the past decade. So believe me when I tell you that this Prime Day is the perfect time to grab a Meta Quest 3, which is .

Read more
Best Samsung Prime Day deals: TVs, phones, monitors and more
Samsung Galaxy Z Fold 5 on a flat surface with the purple Galaxy Buds 2 Pro TWS Bluetooth earbuds on the side.

Samsung is one of the biggest tech conglomerates in the world, and it has one of the biggest ecosystems around, so it makes a lot of sense to go with other Samsung products if you're already one of its users to get that extra integration. Luckily, while Samsung isn't always pricey, there are a lot of Prime Day deals that bring down prices even more, so you can get really affordable stuff, whether you want a new TV or a new smartphone. To that end, we've rounded up our favorite deals across various Samsung product categories and put them for you down below.
Samsung TV Prime Day deals

Samsung is one of the best TV brands, so there will surely be high demand from shoppers for Samsung Prime Day TV deals. However, while Samsung is known for its top-of-the-line TVs with all of the latest technologies, it also offers budget-friendly TVs that are going to be even more affordable during the shopping holiday. Stocks are expected to run out fast though, so if you see a Samsung TV that you want to buy for Prime Day, it's highly recommended that you push through with the transaction as soon as possible.

Read more
This new Xbox Deadpool butt controller raises a lot of questions
A closeup of the Deadpool butt controller. It's red and the back looks like Deadpool's butt cheeks. The text on the image says "Designed by Deadpool xoxo"

Is an Xbox controller with a butt on the back actually comfortable to use? Somebody is about to find out. Xbox announced a limited edition controller on Wednesday that promotes the upcoming Deadpool & Wolverine movie. It's designed to look like Deadpool's backside, complete with a "perfectly rounded tush," according to a post on Xbox Wire.

The accessory is very over the top. The front looks like a mostly regular red Xbox controller with a unique, ridged surface. The two handles, which resemble Deadpool's legs, have straps on them, and if you turn it around, you'll be treated to a pair of prominent buns.

Read more