Ever since generative AI chatbots arrived on the scene, it was obvious that virtual assistants living on our phones will never be the same again. Google kicked off the trend with Gemini, and it seems AI is finally giving some much-needed boost to Siri, as well.
At WWDC 2024, Apple introduced a new avatar for Siri. Borrowing from the skillsets of its rivals, the iPhone’s native AI assistant is finally getting more conversational, alongside superpowers such as summarization and direct integrations with apps.
For example, Siri will now understand your body gestures, such as head movements. To interact with Siri, you can now nod your head to accept an incoming voice call, handle messages, or interact with notifications. Apple says these capabilities are powered by machine learning wizardry facilitated by the onboard H2 chip.
Siri is finally getting a new look, which now shows that it’s active with a brief on-screen edge glow look, instead of the colorful sphere. Siri is also getting more conversation and can accept follow-up queries while preserving the context.
With just a double tap at the bottom of the screen, users can summon Siri without having to summon it with a voice cue. Siri can now answer thousands of questions when you want to know how to do something on your iPhone, iPad, or Mac.
For example, if you ask, “How can I write an email and schedule it for tomorrow?” the assistant will show you instructions right on the screen.
App Intents API is the framework that will allow Siri to interact with apps, both Apple’s own as well as third-party applications. You can tell Siri to pull up photos of your ginger cat, and it will automatically sift through the Photos library and pull up all the relevant images.
Contextual awareness for Siri works across multiple apps in the background. For example, let’s say you’re planning to pick up your brother from school and need to plan it. You can simply tell Siri when your brother will leave school, and the assistant will do the rest of the planning for you by looking at the text exchange and other relevant information saved on your phone.
Apple’s take on AI is called Apple Intelligence, which the company says puts the focus on power, intuitiveness, deep product integration, contextual awareness, and privacy. The motto? “Intelligence that understands you,” says the company.
So, here’s the most important bit: Apple Intelligence will be free. On top of that, it is integrating OpenAI’s ChatGPT, which kicks into action for tasks that Siri can’t accomplish.
On the fun side of things, there is Genmoji. This one lets you create a custom emoji using text prompts right in the keyboard. These can be used as inline emojis, standalone creation, or a tapback response. Think of it as a local text-to-image generator but for fun emojis.
On a similar note, the Photos app gets a new Cleanup tool, which essentially does the same job as Magic Eraser in Google Photos. Apple is also adding a Search in Video tool that lets users find moments in a video clip. Apple Intelligence can also take a prompt from users and automatically generate a movie by combining photos and videos with background audio pulled from the Apple Music library.
On a similar note, users can leverage ChatGPT smarts by having it accomplish tasks like suggesting photo editing prompts and creating original text content, among other tricks. Apple says it will add support for other third-party AI models, as well, down the road.
Apple is touting natural language understanding for Siri, among other products. The onboard AI can rewrite and summarize text content across in-house apps such as Mail. Apple is also focusing on image generation, allowing users to create original media in sketch, illustrations, and animation styles across apps like Messages and Freeform, among others.
And since the focus is on privacy, a healthy share of these AI chores are handled on-device. That means your data never leaves your phone for cloud processing, since all of that happens natively. For more demanding workflows, Apple is touting Private Cloud Compute, a cloud infrastructure built atop Apple silicon that offers the same level of data privacy as the Apple device in your hands.
Apple is also taking cues from Google and bringing Smart Reply to apps like Mail and Messages. Apple Intelligence will also present summaries of email conversations right at the top, somewhat like Shortwave. Apple Intelligence experiences will be rolling out for users later this summer.
