Researchers put A.I. inside a camera lens to compute ‘at the speed of light’

ai camera at the speed of light stanford 0a19e6d7 4f84 46bb 8ba6 b520f06332f8

The camera is the eye for many automated devices and the computer is the brain — but researchers at Stanford University recently combined the two in an attempt to make smart cameras more compact. A team of graduate students recently created an artificially intelligent camera that doesn’t need a large, separate computer to process all the data — because it’s built into the optics itself.

Current object recognition technology uses A.I. on a separate computer to run the images or footage through algorithms to identify objects. As the Stanford researchers explain, driverless cars have a large computer in the trunk in order to recognize when a pedestrian steps out in front of the car’s path. Those computers are big, require lots of energy and are often slow.

ai camera at the speed of light stanford img 3646
The team of Stanford researchers behind the AI camera. Stanford

The researchers instead worked to build the A.I. directly into the camera, both to create smaller systems as well as faster ones. The camera has two layers — the first is called an optical computer. As light passes through the camera, the built-in computer pre-processes that data. The optical computer filters out unnecessary data, reducing the number of necessary calculations. The second layer is a traditional computer and imaging sensor that handles the remaining calculations. The science behind the A.I. camera is complex, but the result could lead to significant advances for devices with built-in cameras like self-driving cars and drones.

By processing the scene as the light hits the camera instead of after the fact, the researchers said they were able to create a camera that didn’t require input power, reducing the typically intense computing power required for object recognition A.I. “We’ve outsourced some of the math of artificial intelligence into the optics,” graduate student Julie Chang said.

ai camera at the speed of light stanford onn1Gordon Wetzstein, the assistant professor that led the research, says that moving the computing to the optics allows for much faster data processing, along with reducing the computing required. “Millions of calculations are circumvented and it all happens at the speed of light,” he said.

While one of the goals is to reduce the size of different devices, the research hasn’t quite reached that point yet. The camera takes up a lab bench, but the researchers suggest that further research will help the concept to decrease in size, eventually making the AI camera small enough for more portable devices. The team suggests several different potential uses, from autonomous cameras and drones to handheld medical imaging. The processing at the speed of light could be a big advantage in self-driving cars that rely on cameras to see and avoid potential collisions.

The device’s size suggests the technology needs some time to mature before actually being integrated into actual products, but the research takes a significant step forward for artificially intelligent cameras. The team recently published the full research in Nature’s Scientific Reports.

Emerging Tech

How emotion-tracking A.I. will change computing as we know it

Affectiva is just one of the startups working to create emotion-tracking A.I. that can work out how you're feeling. Here's why this could change the face of computing as we know it.
Smart Home

Virtually walk through dream homes with Zillow’s new A.I.-powered 3D home tours

Home sellers and real estate agents listing properties on Zillow's home marketplace now can add 3D tours for free to their listings. Zillow 3D Home uses artificial intelligence to create tours with 360-degree panoramic photos.

Vivint’s latest home security camera is infused with artificial intelligence

The Vivint Outdoor Camera Pro uses advanced analytics to differentiate familiar faces from potentially dangerous intruders to keep your home safe and secure when suspicious activity occurs.
Emerging Tech

How MIT hacked horticulture to cultivate a hyper-flavorful basil plant

At MIT, Caleb Harper used his personal food computers to alter the climate in which he grew basil. Exposing it light for 24 hours a day changed the flavor profile of the plant, making it spicier and stronger.

The best budget-friendly GoPro alternatives that won’t leave you broke

Cold weather is here, and a good action camera is the perfect way to record all your adventures. You don't need to shell out the big bucks for a GoPro: Check out these great GoPro alternatives, including some 4K cameras, that won’t leave…

Etch-A-Snap camera puts a modern spin on one of your favorite childhood toys

Can't draw on an Etch A Sketch? Snap a photo with the Etch-A-Snap and the camera will draw out the scene for you. The weirdly cool camera designed by Martin Fitzpatrick replaces the usual LCD screen with an old-school Etch A Sketch.

The Black Eye Pro Cinema Wide G4 is a knockout lens for any smartphone

Where cheaper wide-angle accessory lenses add distortion, and costlier models don't always justify their higher prices, the Black Eye Pro Cinema Wide G4 offers a valuable balance of modest price and high quality optics.

Family feud: Huawei P30 Pro vs. P20 Pro vs. Mate 20 Pro camera shootout

The Huawei P30 Pro's camera has an amazing zoom mode and low light capabilities. But take these away, and how does it compare when facing its sibling phones, the P20 Pro and Mate 20 Pro, taking regular photos?

Nikon Z 7 vs. Sony A7R III: High-res mirrorless cameras compared

The Nikon Z 7 and Sony A7R III both have over 40 megapixels, but which one comes out on top? With similar image quality, the answer comes down to speed, autofocus, battery life, and design.

The Canon EOS Rebel T6 DSLR camera gets a steep price cut at Walmart

Modern smartphones can snap pretty impressive pics, but if you want pro-quality photos, you need a dedicated digital camera. The Canon EOS Rebel T6 is one of the best entry-level DSLR cameras on the market, and it’s on sale right now for…

Panasonic Lumix S1R vs. Nikon Z 7: When megapixels matter, which do you choose?

The 47-megapixels Lumix S1R and 46-megapixel Nikon Z 7 are the two highest-resolution, full-frame mirrorless cameras on the market. The S1R features a high-resolution mode that can take 187MP images, but the Nikon is lighter and cheaper.

Sweet 16: Wacom’s Cintiq 16 pen display makes retouching photos a breeze

Wacom’s Cintiq pen displays are usually reserved for the pros (or wealthy enthusiasts), but the new Cintiq 16 brings screen and stylus editing to an approachable price. Does it cut too much to get there?

China bans selfies at gigantic Aperture Spherical Telescope

You can't take a selfie with the world's largest single-dish radio telescope anymore, as the Chinese government has banned everything from smartphones to digital cameras in the surrounding 5-kilometer area.

Light on price but rich on features, these are the best cameras for students

Need pro-level features on a budget? The best cameras for students mix advanced features with a more palatable price point. From $2K entry level full frame cameras to $600 budget picks, here are five of the best cameras for students.