Apple isn’t immune from the AI craze sweeping the rest of the industry. Following the likes of Google with Gemini Nano, Apple is set to roll out AI upgrades to the iPhone with iOS 18. Code-named “Project Graymatter,” the iOS 18 update will bring a variety of AI-powered enhancements to the iPhone and Siri in particular.
According to AppleInsider, the features are being tested in advance of Apple’s Worldwide Developers Conference (WWDC), and one of the biggest is called “Graymatter Catch Up.” The feature is tied to Siri, Apple’s voice assistant, which will now allow users to request and receive an AI overview of the most recent notifications.
It sounds fairly similar to Amazon Alexa’s daily summaries that you can enable, though Siri will operate as an on-device large language model (LLM). It’ll have access to your other accounts and information, including calendar events, locations, dates, contacts, and companies. We assume this will be optional and can be enabled or disabled since Apple tends to be a bit better about privacy than Google and Samsung.
With the AI enhancements, Siri should be able to summarize all kinds of text, including short messages and long-form articles. Like Google, Apple also intends to add AI-powered text summarization and transcription to its first-party apps and “smart replies” into Messages, Mail, and Siri.
The new and improved Siri should be able to answer queries on the device, summarize long articles, and transcribe audio using the updated Notes and Voice Memos apps. Tasks that can’t be done on the device can be done with Ajax LLM or cloud-based processing. We’ve seen these features on Microsoft’s OneNote and Otter, so it seems clear Apple is trying to compete.
Apple is also looking at some AI-enhanced accessibility features with “more natural” voices and text-to-speech improvements. Last, but not least, Siri should get better cross-device media and TV controls. One of the examples AppleInsider mentions is that users may be able to use Siri on their Apple Watch to play music on another device, though this will likely only roll out late in 2024.
While Siri may be getting the most interesting AI features, other apps aren’t being neglected. For example, iOS 18 and macOS 15 are integrating AI-powered photo editing into the Photos app. Users will be able to use features like “Clean Up” to remove objects and people from image backgrounds, similar to Google’s Magic Eraser. This will replace Apple’s current Retouch tool.
Apple is also expected to introduce something called “Generative Playground,” a generative AI feature that can create and edit images. It should also feature iMessage integration with a dedicated app extension. This could include a variety of images, as well as custom emojis. It’s possible this may be integrated into notes as well.
Last but not least, Math Notes will be integrated into Apple’s new GreyParrot Calculator app. Geared toward students, the Math Notes app seems to recognize mathematical equations and offer solutions for them. It’s essentially an auto-complete feature for math problems rather than text. It will also be able to create graphs, making homework a whole lot easier.
While Project Greymatter sounds impressive, it’s worth noting that, according to Bloomberg, the AI features may be labeled as a beta or a preview when it rolls out. It also seems that, unlike Google Gemini, Ajax LLM won’t be able to generate or process video.
Expect to hear more about the new AI and software features on iOS 18 and macOS 15 when WWDC 2024 rolls around on June 10.
