Skip to main content

Gran Turismo Sophy devs explain how it taught an AI racer sportsmanlike conduct

As the first significant public release of technology from Sony AI, which was formed in 2020, the upcoming Gran Turismo Sophy Race Together mode for Gran Turismo 7 will be many players’ first exposure to complex AI technology that could have a lasting impact on many of Sony Interactive Entertainment’s games. As such, if the AI is rude, unfair, and unbeatable for many players, GT Sophy could be seen as yet another warning sign that AI isn’t ready for primetime yet. That’s why ensuring GT Sophy has proper sports etiquette is a priority for its developers.

In racing, you need to make sure that you’re driving aggressively because you do want to win, but at the same time, you want to drive fairly,” Sony AI COO Michael Spranger tells Digital Trends. “You can’t bump into other cars in order to gain an advantage and need to make sure that everybody can fairly compete. That’s an interesting area because these rules are quite imprecise; there’s a rulebook, but it needs interpretation from judges. Getting this right — driving aggressively and competitively versus driving fairly — is one of the big challenges of GT Sophy.”

Sony AI, Sony Interactive Entertainment, and Polyphony Digital have confirmed that players will be able to race GT Sophy in a limited-time Gran Turismo 7 mode starting on February 21. To get to that point, its developers did a lot of work refining not only GT Sophy’s skill via reinforcement learning but its etiquette toward a nebulous set of courtesy-driven rules as well. Ahead of its addition to Gran Turismo 7, Digital Trends spoke to Sony AI COO Michael Spranger and GT Sophy Project Lead Peter Wurman to learn more about their ethical approach to GT Sophy’s sportsmanlike conduct, and what future uses they see for this technology in gaming.

The following interview has been edited for brevity and clarity. 

What made Gran Turismo the most appealing option to develop an advanced AI for first?

A headshot of Sony AI COO Michael Spranger, PhD.
Michael Spranger, PhD, COO of Sony AI Image used with permission by copyright holder

Michael Spranger: It was because it was possible and because Gran Turismo is a really exciting game. Gran Turismo has been part of PlayStation from the very beginning. It’s an iconic game that provides this specific challenge area of physical realism, strategy, decision-making, and sports etiquette. Also, it has a big audience of people actually following the game, and that means that any technology we develop in the game can have a potentially meaningful impact on millions of players. I also love the game. These things combined made us choose Gran Turismo as one of the first targets for gaming AI projects.

How does reinforcement learning allow this AI to take more risks or drive more impressively than your standard game AI? Is it just trying to drive the most precise racing line? 

Peter Wurman: There are different levels of skill. The baseline is driving faster on the track, so reinforcement learning works by letting the AI agent explore what effect its actions have on the world and then try to maximize its reward. By giving it positive signals for going faster on the track, it learns to drive around the track very fast. It learns to press on the accelerator and break into corners so it doesn’t crash, which works great for training it for a time-trial scenario.

To actually make it a competitive racer, we had to give it other reward signals for passing cars that are in front of it and not preventing cars behind it from passing it. We also gave it penalties for colliding or bumping into other cars. This combination of rewards and penalties took some effort to get right, but once we did training with the right population of opponents, it learned to be a very effective racer.

A headshot of GT Sophy Project Lead and Director of Sony AI America Peter Wurman.
Peter Wurman, PhD, Director of Sony AI America and Project Lead on GT Sophy Image used with permission by copyright holder

What were the biggest learnings from the first couple of Race Together events that you held with top Gran Turismo Sport players?

Peter Wurman: If you’ve followed along with the project, you’ll know there were two races. The first we did well in, but we didn’t win the Team Score. Some of the interesting things we learned from that had to do with finding the balance of assertiveness versus being a good sport. We found out two weeks beforehand that the agent we had trained was too aggressive and wouldn’t be allowed to race, so we had to scramble and adjust things. You can’t have one AI agent that was really good at driving fast and another that thought about the tactical aspects of passing; instead, we had to incorporate them together into one coherent agent that was doing both at the same time. Those are the things we saw after the July 2021 race, so we felt much more confident going into the October 2021 race.

Michael Spranger: There was this thing where we were told it was too aggressive, but then on the first exhibition race day, we actually lost the race because were not aggressive enough and timid. The problem of how you balance those things was really apparent across the development of the project.

A replay of a Gran Turismo Sophy Race Together match in Gran Turismo 7.
Image used with permission by copyright holder

What changes had to be made to GT Sophy when you made the decision to add it to Gran Turismo 7 and not just train it in a test environment? 

Peter Wurman: The biggest is that in all the exhibitions, GT Sophy ran on a separate computer and talked through a networking connection to the PlayStation console. Here, GT Sophy is embedded in the game. Making it work on a PlayStation with the resources that were available to us after all of the rendering and other stuff that’s going on was a bit of engineering work.

The other thing is that Polyphony Digital really wanted the agent to be an extremely good example of racing style and racing skill. We put in some changes to encourage GT Sophy to stay within the lines more. If you’ve watched any of the exhibitions, you’ll see that it was really good at using all of the track, sometimes putting two tires in the grass. That’s legal, but not exactly what you could do in a real race car. We made some changes to keep GT Sophy between the lines more. It’s still very fast but a little cleaner.

We’re mostly focused on trying to adhere to the rules of sportsmanship in automotive racing.

What are the main things the Sony AI team is hoping to learn from adding Gran Turismo Sophy Race Together to Gran Turismo 7?

Peter Wurman: The main one is that we hope people enjoy playing against it, feel like they’re really challenged at all levels of skill, and are learning to become better-simulated racecar drivers.

Michael Spranger: We also want to see how people interact with the technology. Right now, it’s a handful of people who have experienced the technology, so we’re curious to see how people are going to use this to learn for themselves, gain new skills, and have new experiences. I think that’s at the core of the project in many ways. We have the technology breakthrough, and I think that’s really important, but I think we’re unlocking a brand new stage for people and in Gran Turismo. In many ways, the built-in AI has a quite narrow band of performance, and you can surpass that once you become an intermediate driver on each difficulty.

Even with the relatively few track-car combinations that we unveiled here, from what I can do to what some of the top players can do, everybody can find a really interesting match. We hope to see more of that and see how people learn from that. It’s also very difficult for some players to find good opponents, and for the first time, you can have repeated interactions with a worthy opponent without having to call up all your friends and have them all come together. It’s going to be something where people can make a lot of progress in getting better at the game because they can have these tailored experiences to some extent.

A first-person perspective of a Gran Turismo Sophy Race Together race in Gran Turismo 7.
Image used with permission by copyright holder

On the ethical side of things, can you explain the process of determining what constitutes sportsmanlike conduct for GT Sophy so it isn’t harassing certain players?

Peter Wurman: Well, it can’t talk to you, so there are a lot of bad things GT Sophy can’t do because that’s not part of the game. We’re mostly focused on trying to adhere to the rules of sportsmanship in automotive racing; now, those are pretty nebulous. You can’t block a legitimate passing opportunity and things like that. There are a bunch of fuzzy words in these rules that were hard to encode and program, so we had to approximate it with reward signals and penalties if it did things like bump another car.

Then, we had to give it to Polyphony Digital and test drivers and let them give us feedback because racing against other AI agents doesn’t really show us if it’s ready to be racing against the variety of human racing styles. That took some effort and iteration to get that right.

Michael Spranger: For this release, I think it’s fair to say that it is competitive. It is going to try and overtake players, but it’s not going to divebomb them. It’s competitive but polite. It’s definitely putting pressure on you; you’re going to see it in the rearview mirrors trying to overtake you. At the same time, it’s going to try and give you some space so you can race competitively but fairly.

Peter Wurman: But if you make a mess, it’ll take advantage.

Michael Spranger: That’s right, instantly!

Gran Tursimo Sophy Race Together mode's race option menu in Gran Turismo 7.
Image used with permission by copyright holder

Where do you see GT Sophy and AI technology in Sony games going over the next five years?

Michael Spranger: Obviously, we can’t announce anything specifically, but in general, there are multiple layers to it. There are still challenges and things we can do within Gran Turismo to unlock new experiences, and I don’t think we’re finished with that. There’s real excitement about pushing the technology further. Then, from the Sony AI perspective of corporate R&D and strategic initiatives, we’re really excited about what this technology can bring to gaming in general.

Other games will give us a different challenge to overcome and will further the capabilities of the technology. It’s an emerging technology that has had some successes in the past, but I do think what we’re doing here is quite distinct in the sense that we do the scientific breakthroughs but then we also deploy it and make sure that people get their hands on the AI technology. That is going to push the technology forward and unlock greater experiences.

Interaction is at the core of PlayStation, it’s not an error that it’s called Sony Interactive Entertainment. It’s really about what your interaction with the things that you find in the game is, and I do feel that this technology has the potential to really unlock the next generation of games, and we’re really excited about that.

Editors' Recommendations

Tomas Franzese
Gaming Staff Writer
Tomas Franzese is a Staff Writer at Digital Trends, where he reports on and reviews the latest releases and exciting…
The best Bethesda games of all time
Soldiers about to fight with a giant in the background.

Bethesda is one of the rare cases of a game developer finding so much success that it branched out to become one of the biggest publishers in the Western gaming market. While not quite as gigantic as the likes of Activision or EA, Bethesda has a great stable of developers, itself included, that have made some of the most ambitious games on the market. Whether it's the cutting-edge technology coming out of ID Software, the high-octane FPS action of Machine Games, or even the more experimental titles from Arkane, Bethesda's name is attached to far more than just its high-profile RPG series.

Since it was acquired by Microsoft, all of Bethesda's games, including the games made by the studios under it, will be console-exclusive to the X-box platform moving forward. However, there is a long history of games already released for multiple platforms, and between all the studios Bethesda is in charge of, there are games of all types to check out. You will obviously know the major players here, such as The Elder Scrolls and Fallout, but there are way more Bethesda games that are just as good. That's why we've rounded up this list of the best Bethesda games of all time.

Read more
Best gaming PC deals: Powerful gaming rigs from $600
young woman playing video games on a PC

A desktop PC offers a lot that even the best laptops can’t offer, such as expanded connectivity and the ability to swap upgraded components in the future. These features are especially handy if you’re a gamer. The best gaming PCs are a worthy investment if that’s the case, and it’s always worth searching through the best gaming PC deals if you’re in the market for an upgrade, or if you’re building a gaming setup from scratch. There are a lot of great gaming PC deals currently available, and they include brands like HP, Alienware, Lenovo, and Asus. We’ve rounded them all up, so give them a look and throw the savings at some of the best PC games.
HP OMEN 25L — $600, was $900

The HP OMEN 25L is one of the most popular gaming PCs on the market, as it offers some great value. Like all of the best desktop computers, it’s fully customizable, meaning you can purchase it at this great price and upgrade internal components if you need to in the future. This build has some good specs as it is, however, offering an Intel i3 processor, 8GB of RAM, and an NVIDIA GeForce GTX 1660 graphics card. This is a great option for gamers on a budget, and it’s more than enough hardware to get you going if you’re new to the gaming world.

Read more
This deal gets you a PS5 for less than $470
The Playstation 5 system standing upright.

One of the best PlayStation deals is perfect if you don't mind buying a refurbished model. Over at Back Market, you can buy the PlayStation 5 for only $463 for a good-condition console. It's a refurbished model so it's been previously owned, however, it's verified to be fully working and comes with a one-year warranty so it's not much different compared to buying new. Your console will be repaired or replaced in the first year if the worst happens. Sounds appealing? Keep reading while we tell you more about the PlayStation 5.

Why you should buy the PlayStation 5
The Sony PlayStation 5 meets the expectations we all have of the latest game consoles. It loads games within mere seconds with the best PS5 games being something truly special and including titles like Marvel's Spider-Man: Miles Morales, Gran Turismo 7, and God of War: Ragnarok.

Read more