Google is bringing AI superpowers to your smartphone camera, and it looks ridiculous

By Nadeem Sarwar Published May 14, 2024

Google Astra on a phone. — Google

It seems AI assistants are antique, or that’s what Google wants you to believe, for we are in the era of AI agents — and Google I/O 2024 has quickly proven that. Say hello to Project Astra, a generative AI agent with vision, text, and speech capabilities, with a sprinkling of memory and spatial awareness capabilities in tow.

Think of it as eyes for your phone that can make sense of the world around you. Point it at a mathematical equation, and it will solve it. Pointing the camera at a cat? Astra will suggest an apt name for the feline meow-ster. Ask it where you left your earbuds, and if the camera sensor has seen them, it will say something like, “You left them on the sofa.”

Recommended Videos

Astra can make sense of code appearing on a screen, identify objects and explain what they do, identify buildings, and more. Think of it as Google Lens but for the entire world, and can make sense of most anything in front of the camera’s lens.

Unlike Google Assistant, you don’t need to prompt it. Just point the camera at anything, utter the audio query, and Astra will explain in a natural language response. Google says Project Astra will be rolled out via the Gemini app later this year.

An all-seeing AI agent?

Project Astra demonstration on a phone. — Google

If Project Astra sounds familiar, that’s because OpenAI demoed a similar feature for ChatGPT — powered by the new GPT-4o model — just a day ago. OpenAI’s tool is currently under the red-teaming phase for safety testing and will be released in a phased manner, starting with ChatGPT Plus subscribers.

Google won’t say whether Astra will have a price tag. But given the near spontaneity of responses, the kind of visual data crunching involved, and the generative chops required to offer a meaningful response, it’s unlikely Astra will be served as a free perk owing to the compute requirements.

The best example would be the Google One AI Premium subscription, which already hides some of the new Gemini-powered experiences behind a paywall. But so far, Astra looks like the most amazing AI innovation Google has showcased ever since it wowed the world with Duplex’s capabilities a few years ago.

Managing Editor

Nadeem is the Managing Editor at Digital Trends.

Topics

Phones

Your backpack needs one of these power banks before classes start

I picked 4 power banks that can survive your longest day on campus

Anker Nano 3-in-1 10,000mAh portable power bank for iPhone.

Finding an empty outlet on campus can require more patience than registering for a popular class. Common spaces like libraries or lecture halls rarely have sockets in the right places. So a good power bank gives you far more freedom. It gives you the option to choose the quiet corner of the library, work outside between classes, or even finish an assignment without worrying about your laptop's battery.

The right model depends on what you carry, since a phone needs far less power than a MacBook or gaming laptop. I picked four Anker power banks that cover everything from everyday phone charging to running several demanding devices at once.

Phones

The Minimal Phone 2 looks dramatically better than the original, and I’m genuinely here for it

Plastic is out, metal is in, and Minimal Phone 2's renders suggest the digital detox phone concept finally has the hardware to match its ambition.

Electronics, Mobile Phone, Phone

I've been following the Minimal Phone since its initial crowdfunding days. The first iteration had the right idea trapped inside the wrong body, I’d say. Plastic build, no 5G, an underwhelming camera, and e-paper ghosting that frustrated early adopters; all these issues kept the phone from reaching a wider audience.

Now, founder Armen Youssefian has posted what appears to be a 360-degree render of the Minimal Phone 2 on X, and from what I can see, the team has done its homework.

Phones

The EU says TikTok still isn’t doing enough to protect kids

Regulators say the platform's privacy settings don't go far enough under the Digital Services Act.

TikTok Creative Featured

The European Commission has accused TikTok of falling short on protecting minors, saying the platform's current account settings don't meet the child safety standards required under the Digital Services Act (DSA). The findings are preliminary, but if upheld, TikTok could face a fine of up to 6% of its global annual revenue.

Why the EU thinks TikTok isn't doing enough