“If it only works 30 percent of the time, it’s enough that people will start using it.”
The system, called Im2Calories, uses a combination of image recognition and comparative analysis to identify meals from average-definition photos. As Google research scientist Kevin Murphy demonstrated at this week’s Rework Deep Learning Summit in Boston, the system determines the depth of each pixel in an image, matches the results to a vast database of nutritional information, and then takes into account portions by gauging the size of the food relative to the plate it’s on. In one test, Murphy said, Im2Calories was able to calculate the accurate caloric total of two eggs, two pancakes, three strips of bacon, and the accompanying condiments.
Im2Calories isn’t perfect, though. It currently includes a dropdown menu for users to select the correct food when it misidentifies something, which happens somewhat frequently right now. However, Murphy told Popular Science that complete accuracy is at this point unrealistic.
“Ok fine, maybe we get the calories off by 20 percent. It doesn’t matter. We’re going to average over a week or a month or a year,” he said. Recognition can only improve as the body of data grows, he said. “If it only works 30 percent of the time, it’s enough that people will start using it, we’ll collect data, and it’ll get better over time.”
Of course, that’s only true if the nutrition labels themselves are correct. The New York Times reports that the current system of measuring calories — burning food to see how much energy it contains, a method developed in the 19th century — doesn’t take into account digestion. The body expends more energy to break down high-protein foods like meat and yogurt, and some foods aren’t digested at all. The labels on nuts are probably the most misleading — according to some researchers, they’re likely 25 percent too high.
“If we can do this for food, that’s just the killer app.”
And that’s to say nothing of the controversy around the practice of calorie tabulating in the first place. A 2011 Harvard University study published in the New England Journal of Medicine found that the makeup of food was a far bigger determinant of health than its caloric value.
“This study shows that conventional wisdom — to eat everything in moderation, eat fewer calories and avoid fatty foods — isn’t the best approach,” Dr. Dariush Mozaffarian, a cardiologist and epidemiologist at the Harvard School of Public Health and lead author of the study, told the New York Times in an interview. “What you eat makes quite a difference. Just counting calories won’t matter much unless you look at the kinds of calories you’re eating.”
Perhaps that’s why Murphy floated the idea of applying the mechanics behind Im2Calories — automated object recognition — to other fields. “If we can do this for food, that’s just the killer app,” he said. “We can do things like traffic scene analysis, predict where the most likely parking spot is. And since this is all learned from data, the technology is the same, you just change the data.”
Murphy wasn’t forthcoming about a release timetable for Im2Calories, or with details of the form it could take. Those details are likely months to years from being hashed out, but when Im2Calories does launch publicly, prepare to feel a lot guiltier about your sugary indulgences.
