Skip to main content

Computer scientists develop AI that gets curious about its surroundings

Curiosity Driven Exploration by Self-Supervised Prediction
Artificial intelligence is showing a greater range of abilities and use-cases than ever, but it’s still relatively short on desires and emotions. That could be changing, however, courtesy of research at the University of California, Berkeley, where computer scientists have developed an AI agent that’s naturally (or, well, as naturally as any artificial agent can be) curious.

In tests, they set the AI playing games such as Super Mario and a basic 3D shooting game called VizDoom, and in the games, it displayed a propensity for exploring its environment.

Recommended Videos

“Recent success in AI, specifically in reinforcement learning (RL), mostly relies on having explicit dense supervision — such as rewards from the environment that can be positive or negative,” Deepak Pathak, a researcher on the project, told Digital Trends. “For example, most RL algorithms need access to the dense score when learning to play computer games. It is easy to construct a dense reward structure in such games, but one cannot assume the availability of an explicit dense reward-based supervision in the real world with similar ease.”

But given that Super Mario is — last time we checked — a game, how does this differ from AI like the DeepMind artificial intelligence that learned to play Atari games? According to Pathak, the answer is in its approach to what it is doing. Rather than simply trying to complete a game, it sets out to find novel things to do.

“The major contribution of this work is showing that curiosity-driven intrinsic motivation allows the agent to learn even when rewards are absent,” he said.

This, he notes, is similar to the way we show curiosity as humans. “Babies entertain themselves by picking up random objects and playing with toys,” Pathak continued. “In doing so, they are driven by their innate curiosity, and not by external rewards or the desire to achieve a goal. Their intrinsic motivation to explore new, interesting spaces and objects not only helps them learn more about their immediate surroundings, but also learn more generalizable skills. Hence, reducing the dependence on dense supervision from the environment with an intrinsic motivation to drive progress is a fundamental problem.”

Although it’s still relatively early in the project, the team now wants to build on its research by applying the ideas to real robots.

“Curiosity signal would help the robots explore their environment efficiently by visiting novel states, and develop skills that could be transferred to different environments,” Pathak said. “For example, the VizDoom agent learns to navigate hallways, and avoid collisions or bumping into walls on its own, only by curiosity, and these skills generalize to different maps and textures.”

Luke Dormehl
Former Digital Trends Contributor
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
AI breakthroughs could come via the brains of bees, scientists say
ai bee brain opteran

The brains of bees could help to take AI systems to the next level, according to scientists in the U.K.

The team at the University of Sheffield has conducted a study that it says reveals the underlying mechanisms that drive the creatures' ”remarkable” decision-making capabilities, which could be transferred to AI technologies, the BBC reported.

Read more
OpenAI reveals location of its first international outpost
ChatGPT app running on an iPhone.

ChatGPT creator OpenAI is set to open an office in London in what will be its first site outside of the U.S.

The Microsoft-backed startup has gained much attention since November's launch of its AI-powered ChatGPT chatbot, the text-based tool impressing many with its remarkable ability to perform a range of tasks and converse in a human-like way.

Read more
Senators to get AI lessons ahead of regulation decisions
The ChatGPT website on an iPhone.

Senate Majority Leader Chuck Schumer has set up a series of briefings aimed at educating senators about artificial intelligence (AI), insisting that his colleagues must "deepen our expertise in this pressing topic."

The move comes as U.S. lawmakers consider how to regulate the new wave of fast-evolving AI technology behind powerful chatbot tools such as OpenAI’s ChatGPT and Google’s Bard.

Read more