Skip to main content

Apple opens digital journal to showcase its machine learning developments

Apple opened a new digital journal to showcase some of the developments it is making in the field of machine learning. In the first entry, it explains what it is doing to help improve the realism of synthetic images, which can, in turn, be used to teach algorithms how to classify images, without needing to painstakingly label them manually.

One of the biggest hurdles in artificial intelligence is teaching it things that humans take for granted. While you could conceivably hand-program an AI to understand everything, that would take a very, very long time and would be nigh on impossible to power. Instead, machine learning lets us teach algorithms much like you would a human, but that requires specialist techniques.

Apple

When it comes to teaching how to classify images, synthetic images can be used, but as Apple points out in its first blog post, that can lead to poor generalizations, because of the low quality of a synthetic image. That is why it’s been working on developing better, more detailed images for machines to learn from.

Recommended Videos

Although this is far from a new technique, it has traditionally been a costly one. Apple developed a much more economical “refiner” which is able to look at unlabeled real images and reference them to refine synthetic images into something much closer to reality.

However, how do you select the correct real image to give the refiner a strong source material to base its refinements on? That requires a secondary image identifier, known as the discriminator. It goes back and forth with the refiner attempting to “trick” the discriminator by gradually building up the synthetic image until it possesses far more of the details of the real images. Once the discriminator can no longer properly categorize them, the simulation halts and moves on to a new image.

Apple

This teaches both the discriminator and the refiner while they compete, thereby gradually enhancing the tools as they build up a strong library of detailed synthetic images.

The learning process is a detailed one, with Apple going to great lengths to preserve original aspects of images while avoiding the artifacts that can build up during image processing. It is worth it though, as further testing has shown vastly improved performance for image categorization based on refined synthetic images, especially when they have been refined multiple times.

Jon Martindale
Jon Martindale is a freelance evergreen writer and occasional section coordinator, covering how to guides, best-of lists, and…
Apple hit with lawsuit over Apple Intelligence delay
Invoking Siri on iPhone.

Apple has been hit with a lawsuit over allegations of false advertising and unfair competition regarding the delayed launch of some of its Apple Intelligence features.

The tech company has made much of its AI-infused Apple Intelligence tools when they were first unveiled at its developer event in June 2024, and while some of the features have made their way to its various devices since then, the company recently revealed that some of the more advanced AI-powered tools -- including for its Siri virtual assistant -- would not be ready until 2026.

Read more
Apple’s hardware can dominate in AI — so why is Siri struggling so much?
Apple's Craig Federighi presents the Image Playground app running on macOS Sequoia at the company's Worldwide Developers Conference (WWDC) in June 2024.

Over the past year or so, a strange contradiction has emerged in the world of Apple: the company makes some of the best computers in the world, whether you need a simple consumer laptop or a high-powered workstation. Yet Apple’s artificial intelligence (AI) efforts are struggling so much that it’s almost laughable.

Take Siri, for example. Many readers will have heard that Apple has taken the highly unusual (and highly embarrassing) step of publicly admitting the new, AI-backed Siri needs more time in the oven. The new Siri infused with Apple Intelligence just isn’t living up to Apple’s promises.

Read more
HuggingSnap app serves Apple’s best AI tool, with a convenient twist
HuggingSnap recognizing contents on a table.

Machine learning platform, Hugging Face, has released an iOS app that will make sense of the world around you as seen by your iPhone’s camera. Just point it at a scene, or click a picture, and it will deploy an AI to describe it, identify objects, perform translation, or pull text-based details.
Named HuggingSnap, the app takes a multi-model approach to understanding the scene around you as an input, and it’s now available for free on the App Store. It is powered by SmolVLM2, an open AI model that can handle text, image, and video as input formats.
The overarching goal of the app is to let people learn about the objects and scenery around them, including plant and animal recognition. The idea is not too different from Visual Intelligence on iPhones, but HuggingSnap has a crucial leg-up over its Apple rival.

It doesn’t require internet to work
SmolVLM2 running in an iPhone
All it needs is an iPhone running iOS 18 and you’re good to go. The UI of HuggingSnap is not too different from what you get with Visual Intelligence. But there’s a fundamental difference here.
Apple relies on ChatGPT for Visual Intelligence to work. That’s because Siri is currently not capable of acting like a generative AI tool, such as ChatGPT or Google’s Gemini, both of which have their own knowledge bank. Instead, it offloads all such user requests and queries to ChatGPT.
That requires an internet connection since ChatGPT can’t work in offline mode. HuggingSnap, on the other hand, works just fine. Moreover, an offline approach means no user data ever leaves your phone, which is always a welcome change from a privacy perspective. 

Read more