Skip to main content

A.I. can generate pictures of a finished meal based on just a written recipe

Recipe books contain photos. You know why? Because most of us aren’t capable of reading through a list of ingredients and associated instructions and picturing exactly what is supposed to come out of the oven or other cooking appliance at the end of the process. Artificial intelligence, it seems, doesn’t have quite the same problem — at least, not according to a project carried out by researchers from Tel-Aviv University in Israel.

Using a training data set of approximately 52,000 written recipes, along with images showing the completed foods, the researchers were able to devise a system that can read a recipe and then generate a picture showing what the end result is likely to look like.

“Our system takes a recipe as an input and generates, from scratch, an image that reflects the food that the system ‘believes’ this recipe describes,” Ori Bar El, one of the co-authors on the paper, told Digital Trends. “The important aspect is that the system has no access to the title of the recipe — otherwise this task would have been pretty easy — and that the text of the recipe is both long and does not describe the visual content of the image directly. [This fact] makes this task very hard even for humans, and all the more so for computers.”

Image used with permission by copyright holder

The neural network responsible for the feat generates its images using a two-stage process. First, the text of the recipe is converted into a vector of numbers in a process called text embedding. This numerical representation attempts to capture the meaning of the text by mapping semantically similar pieces of text to close vectors in the embedding space. After this is done, a separate network maps the text vectors and images to align them.

In the second stage, the team uses a Generative Adversarial Network (GAN) which both generates new images and evaluates them. This is the process that resulted in the A.I.-created painting which sold at Christie’s auction last year. By having the GAN attempt to fool itself into thinking a generated image is a real photo, the pictures the system comes up with look increasingly realistic.

“[One] challenge we faced was the fact that the quality of the images in the dataset we used was low,” Bar El continued. “This is reflected by lots of blurred images with bad lighting conditions.” The system also turned out to be better at generating certain, more formless foods (pasta, rice, soups, and salads) than others that had a distinctive shape, such as hamburgers.

While the results may be quite good enough for sharing on Instagram, however, it’s nonetheless an impressive example of machine learning. Pair it with IBM’s recipe-generating Chef Watson and it would be more dazzling.

Editors' Recommendations

Luke Dormehl
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
This basic human skill is the next major milestone for A.I.
Profile of head on computer chip artificial intelligence.

Remember the amazing, revelatory feeling when you first discovered the existence of cause and effect? That’s a trick question. Kids start learning the principle of causality from as early as eight months old, helping them to make rudimentary inferences about the world around them. But most of us don’t remember much before the age of around three or four, so the important lesson of “why” is something we simply take for granted.

It’s not only a crucial lesson for humans to learn, but also one that today’s artificial intelligence systems are pretty darn bad at. While modern A.I. is capable of beating human players at Go and driving cars on busy streets, this is not necessarily comparable with the kind of intelligence humans might use to master these abilities. That’s because humans -- even small infants -- possess the ability to generalize by applying knowledge from one domain to another. For A.I. to live up to its potential, this is something it also needs to be able to do.

Read more
A.I. can tell if you’re a good surgeon just by scanning your brain
brain with computer text scrolling artificial intelligence

Could a brain scan be the best way to tell a top-notch surgeon? Well, kind of. Researchers at Rensselaer Polytechnic Institute and the University at Buffalo have developed Brain-NET, a deep learning A.I. tool that can accurately predict a surgeon’s certification scores based on their neuroimaging data.

This certification score, known as the Fundamentals of Laparoscopic Surgery program (FLS), is currently calculated manually using a formula that is extremely time and labor-consuming. The idea behind it is to give an objective assessment of surgical skills, thereby demonstrating effective training.

Read more
This A.I. meme generator has mastered the art of oddball internet humor
this meme does not exist generator woman yelling at cat

It sounds kind of silly to say, but memes are one of the things that make us human. They are, by design, massively shareable images based on some universal (or at least semi-universal) lived experience that draws on some aspect of popular culture, but seeds it with additional meaning.

Against that backdrop, how could some smartass not come up with a way to hand the job of meme creation over to an A.I.? Heck, robots are threatening to take a bunch of our most important jobs. Why not the task of creating new pithy Spongebob Squarepants images alongside the others?

Read more