Skip to main content

Qualcomm wants to add these crazy AI tools to your Android phone

Qualcomm Snapdragon 8 Gen 3 media asset.

At Mobile World Congress 2024, Qualcomm is adding more to its portfolio of AI-on-phone tricks facilitated by the Snapdragon series silicon for Android phones.  The chipmaker has already showcased some impressive AI capabilities for the Snapdragon 8 Gen 3 flagship, such as voice-activated media editing, on-device image generation using Stable Diffusion, and a smarter virtual assistant built atop large language models from the likes of Meta.

Today, the company is adding more grunt to those AI superpowers. The first is the ability to run a Large Language and Vision Assistant (LLaVa) on a smartphone. Think of it as a chatbot like ChatGPT that has been granted Google Lens abilities. As such, Qualcomm’s solution can not only accept text input, but also process images.

For example, you can push an image depicting a charcuterie board and ask questions based on it. The AI assistant, based on a large multimodal model (LMM) that can process over 7 billion parameters, will then tell you all the kinds of fruits, cheeses, meats, and nuts on the board depicted in the input image seen below.

Qualcomm’s demo of smarter AI assistant on phone.

It can also handle follow-on queries, so you can conduct a flowing back-and-forth conversation. Now, the likes of ChatGPT have also gained multiple-modal capabilities, which means OpenAI’s tool can also process image inputs. However, there’s a crucial difference.

Products like ChatGPT and Copilot are still very much tethered to a cloud-based architecture, meaning your data is handled on remote servers. Qualcomm’s push is in the direction of on-device processing. Everything happens on your phone, which means the whole process is faster, and there is little risk of privacy intrusion.

“This LMM runs at a responsive token rate on device, which results in enhanced privacy, reliability, personalization, and costs,” says Qualcomm. Whether Qualcomm’s promised LLaVa-based virtual assistant will arrive as a standalone app or if it will carry a fee is yet to be officially confirmed.

The next announcement from Qualcomm dives into the creative domain of image generation and manipulation. Not too long ago, Qualcomm demoed the world’s fastest text-to-image generation on a phone using Stable Diffusion tech. Today, the company is giving a first glimpse of LoRA-driven image generation.

Qualcomm showcase of AI image generation on phone.

LoRA takes a different approach to image generation than a regular generative AI tool such as Dall.E. LoRA, short for Low-Rank Adaptation, is a technique developed by Microsoft. Training an AI model can be quite cost-prohibitive, high on latency, and particularly demanding from a hardware perspective.

What LoRA does is it dramatically reduces the model weight, a goal that is achieved by only focusing on specific segments of the model and reducing the number of parameters for training purposes. In doing so, the memory requirements go down, the process becomes faster, and the amount of time and effort it takes to adapt a text-to-image model also drops dramatically.

Over time, the LoRA distillation technique has been applied to the Stable Diffusion model for generating images from text prompts. Owing to the gains in efficiency and the easier adaptability of LoRA-based models, it is seen as a tailor-made route for smartphones. Qualcomm certainly thinks so, and even rival MediaTek has embraced the same solution for generative AI tricks on its flagship Dimensity 9300 chip.

Qualcomm is also showcasing a few other AI tricks at MWC 2024, some of which have already appeared on the Samsung Galaxy S24 Ultra. Among them is the ability to expand the canvas of an image using generative AI fill and AI-powered video generation. The latter is quite ambitious, especially after seeing what OpenAI has accomplished with Sora. It would be interesting to see how Qualcomm manages to port it over to smartphones.

Editors' Recommendations

Nadeem Sarwar
Nadeem is a tech journalist who started reading about cool smartphone tech out of curiosity and soon started writing…
There’s something Samsung didn’t tell you about the Galaxy S24
The Samsung Galaxy S24 Ultra in its launch colors.

“Look, dude, I don’t know how they are going to manage the costs of licensing from AI companies and cloud partnerships, among other associated factors. Nothing comes free.” That’s what a machine learning engineer told me a few days ago when I explained to him how Qualcomm and MediaTek are bringing some neat generative AI tricks to phones.

Well, Samsung has confirmed those fears and quietly dropped the bombshell that at least some of its snazzy AI tricks for the Galaxy S24 series phones will eventually ask you to cough up some cash. That’s going to happen next year, but we don’t know how much you'll have to pay and in what way — at least not right now.

Read more
This cute AI gadget wants to replace your smartphone
Photo of the Rabbit R1.

“Infer and model human actions on computer interfaces by learning users’ intention and behavior when they use specific apps, and then mimic and perform them both reliably and quickly.” That’s the promise of a rather cute, but cutting-edge device called the Rabbit R1, which was previewed at CES 2024. In simpler terms, it wants to keep us from getting lost in the maze of smartphone apps.

Instead, it wants to replicate human interactions with apps by learning and then removing them from the equation. And it can do it all without requiring a phone to pair with. Cellular connectivity is part of the package here, as is Wi-Fi, to execute AI-based tasks within apps without actually having to open those apps on your phone.

Read more
Qualcomm’s newest chip will bring AI to cheaper Android phones
Qualcomm Snapdragon 7 Gen 3 artwork.

Qualcomm has a new mobile platform on the table, and this one targets upper-midrange smartphones and promises to bring some new AI tricks. The latest from the chipmaker is the Snapdragon 7 Gen 3, which technically succeeds the Snapdragon 7+ Gen 2, but the company is comparing most of the improvements against the older Snapdragon 7 Gen 1. 
The new platform is said to bring a 15% boost in processing power, a 20% rise in energy efficiency, and a massive 50% jump in graphics capabilities. Based on the 4nm fabrication process, it packs a single prime core, a trio of performance cores, and four efficiency cores. Interestingly, these cores are clocked at a lower frequency compared to those on the Snapdragon 7+ Gen 2. However, this won't be the only area where Qualcomm's latest sounds like a mixed bag.
Qualcomm says the new chip improves AI-assisted face detection accuracy, but it adds that AI also lends a hand at tasks like making sense of routines and how users interact with apps. There are also a handful of new software-side enhancements coming to the Snapdragon Gen 7 series for the first time. 
Those include an AI re-mosaicing system for reducing grainy textures in photos, bringing down noise, and video retouching. Support for Ultra HDR is also a first for the midrange chip. Spatial audio with head tracking and CD-quality wireless audio are a part of the package as well.

The Snapdragon 7 Gen 3 jumps to the X63 cellular modem that promises a higher downlink speed of up to 5Gbps. Interestingly, it adopts the Fast Connect 6700 Bluetooth + Wi-Fi modem instead of the speedier Fast Connect 6900 modem on the Snapdragon 7+ Gen 2. 
The camera capabilities situation is also interesting. The Snapdragon 7 Gen 3 relies on a triple 12-bit ISP system, while the Snapdragon 7+ Gen 2 puts its trust in a more advanced triple 18-bit ISP architecture. The latter allows higher-resolution photo and video capture in single and dual camera configurations. 
In fact, the Snapdragon 7 Gen 3’s ISP steps down to 120 frames-per-second (fps) slo-mo video capture compared to the 1080p 240 fps video recording allowed by its direct predecessor. Overall, it seems like Qualcomm jumped into its parts bin and crafted a half-new midrange chip for Android phones.
Qualcomm says China’s Vivo and Honor are the first adopters of the Snapdragon 7 Gen 3. The first wave of phones powered by the new chip is expected to be announced later this month. 

Read more