Imagine if you were able to point your smartphone, tablet or augmented reality (AR) headset at a still image, and watch it come to life before your very eyes. This is something that’s possible using the latest augmented reality technology, such as Apple’s ARKit or Google’s ARCore. However, while we have seen it done before, typically it has been limited to just one or two select images — for instance, as a tool for animating advertising billboards.
Researchers from the University of Washington and Facebook have built something that’s a whole lot more versatile. Their new “Photo Wake-Up” project will animate stationary characters in any image, resulting in them springing into action to sprint toward you out of the background. Because who has time for boring old still images as we reach the final days of 2018?
The researchers write that: “The key contributions of this paper are [firstly], an application of viewing and animating humans in single photos in 3D. [Secondly], a novel 2D warping method to deform a posable template body model to fit the person’s complex silhouette to create an animatable mesh. And [thirdly], a method for handling partial self occlusions. We compare to state-of-the-art related methods and evaluate results with human studies. Further, we present an interactive interface that allows re-posing the person in 3D, and an augmented reality setup where the animated 3D person can emerge from the photo into the real world.”
As seen in the video up top, this effect was successfully demonstrated on photographs, in posters, and in art work. At present, the research is still at a relatively early stage, although it proves impressively adept at recognizing human figures across a wide range of mediums, regardless of whether they are modernist painted figures in a Picasso painting or ones taken from a black and white photograph.
In their paper, the researchers suggest that this work could open up new ways to “enjoy and interact” with images. If similar interactions could be extended to, say, making photographic cars drive, clouds move and any other number of possibilities, this could turn out to be the start of something very exciting. As it is, we could certainly see it saving animators considerable time.
The paper, titled “Photo Wake-Up: 3D Character Animation from a Single Photo,” is available to read online.
