On Wednesday, DeepMind released a set of tools for researching and building artificial intelligence (AI) using StarCraft II as a learning environment. Blizzard first revealed DeepMind’s plans to use the popular sci-fi strategy game at the end of 2016, and said it was currently creating a rich application programming interface (API) just for training AI “agents” within the game.
The new StarCraft II-based toolset includes Blizzard’s Machine Learning API, a data set containing game replays performed by up to 500,000 anonymous users, and an open-source version of DeepMind’s PySC2 toolset. The kit also includes mini-games for developers to test their AI for specific tasks, and a joint paper listing baseline results along with an outline of the environment.
The lure of StarCraft II is that the game makes players really think. It’s all about strategy and defeating a single opponent by managing troops, resources, building equipment and bases, and conquering territories. It’s a war between two minds, starting from selecting a race to dominating the map and defeating the opponent.
AI “agents” (or bots) will approach StarCraft II just like any human. The Ai won’t have access to the back-end, it won’t be able to view the entire playing field from the start, and it won’t have any kind of software-based advantage that in the human world would be considered as cheating. AI agents will be required to make the same decisions in race selection, resource gathering, defense management, and in exploring unseen areas.
“The game also has other qualities that appeal to researchers, such as the large pool of avid players that compete online every day,” DeepMind said. “This ensures that there is a large quantity of replay data to learn from – as well as a large quantity of extremely talented opponents for AI agents.”
Another StarCraft II lure is the number of actions an AI agent can take. For instance, old Atari games only have around 10 different actions, such as up, down. left, right, and so on. In StarCraft II, the game provides more than 300 basic actions an AI agent can take. But according to DeepMind, there are around 100 million possible actions available on a screen measuring just 84 x 84.
As for the mini-games, these can be used by trained and untrained agents. They are essentially “assignments” inside StarCraft II used to see how an agent performs a specific task using a specific technique, such as selecting units, collecting minerals, and managing the camera.
Right now, there’s still progress to be made in terms of agents beating the game. DeepMind said that the strongest of baseline agents, A3C, can’t seem to beat the game’s built-in artificial opponent. According to DeepMind, in order for agents to be competitive, there needs to be a breakthrough in deep trial-and-error learning, and “other areas.”
“Our hope is that the release of these new tools will build on the work that the AI community has already done in StarCraft,” DeepMind said, “encouraging more DeepRL research and making it easier for researchers to focus on the frontiers of our field.”
