Meta made DALL-E for video, and it’s both creepy and amazing

By David Matthews September 29, 2022

Meta unveiled a crazy artificial intelligence model that allows users to turn their typed descriptions into video. The system is called Make-A-Video and is the latest in a trend of AI generated content on the web.

The system accepts short descriptions like “a robot surfing a wave in the ocean” or “clown fish swimming through the coral reef” and dynamically generates a short GIF of the description. There are even three different styles of videos to choose from: surreal, realistic, and stylized.

An artist’s brush painting on a canvas close up

According to a Facebook post by Meta CEO, Mark Zuckerberg, translating written text into video is much harder because of how video requires movement:

“It’s much harder to generate video than photos because beyond correctly generating each pixel, the system also has to predict how they’ll change over time. Make-A-Video solves this by adding a layer of unsupervised learning that enables the system to understand motion in the physical world and apply it to traditional text-to-image generation.”

A young couple walking in a heavy rain

Meta’s AI Research team wrote a paper describing how the system works and how it differs from current text-to-image (T2I) methods. Unlike other machine language models, Meta’s Text-to-Video (T2V) method doesn’t use pre-defined text-video pairs. For example, it doesn’t pair “man walking” with a video of an actual man walking.

If this sounds a lot like DALL-E, the popular T2I application, you wouldn’t be far off. Other T2I applications have rolled out since DALL-E gained popularity. TikTok released a filter in August called AI Greenscreen that generates painting style images based on the words you type.

A fluffy baby sloth with an orange knitted hat trying to figure out a laptop close up highly detailed studio lighting screen reflecting in its eye

AI-generated content has become quite buzzworthy within the last few years. Deepfake technology, machine learning techniques to replace a person’s face with another, is even used by visual effects studios for big budget shows like The Mandalorian.

In July, The Times mistakenly reported on a Ukrainian woman in the midst of the Russia-Ukraine war. The problem is she wasn’t real.

The threat of AI probably isn’t a real threat, but projects like DALL-E and Make-A-Video are fun explorations into some of the interesting possibilities.

Editors' Recommendations

Topics

Computing Writer

David is a freelance journalist based just outside of Washington D.C. specializing in consumer technology and gaming. He has…

Computing

Photoshop AI thinks ‘happiness’ is a smile with rotten teeth

Phil Nickinson, as edited by Adobe Photoshop's Neural Filter.

You can't swing a dead cat these days without running into AI. And nowhere is that more true than in photography. I've certainly had fun with it on more than my share of photos. But the more I attempt to be a "serious" photographer, the less I want to rely on artificial intelligence to do my job for me.

That's not to say it doesn't have its place. Because it does. And at the end of the day, using AI filters isn't really any different than hitting "auto" in Photoshop or Lightroom and using those results. And AI certainly has its place in the world of art. (Though I'd probably put that place somewhere way in the back, behind the humans who make it all possible in the first place.)

Computing

I’ve seen the (distant) future of AI web search – here’s where it’s amazing, and where it struggles

Bing copilot AI chat interface.

The aggressiveness with which artificial intelligence (AI) moved from the realm of theoretical power into real-world consumer-ready products is astonishing. For several years now, and up until a couple of months ago when OpenAI's ChatGPT broke onto the scene, companies from the titans of Microsoft and Google down to myriad startups espoused the benefits of AI with little practical application of the tech to back it up. Everyone knew AI was a thing, but most didn't actually utilize it.

Just a handful of weeks after announcing an investment in OpenAI, Microsoft launched a publicly-accessible beta version of its Bing search engine and Edge browser powered by the same technology that has made ChatGPT the talk of the town. ChatGPT itself has been a fun thing to play with, but launching something far more powerful and fully integrated into consumer products like Bing and Edge is an entirely new level of exposure for this tech. The significance of this step cannot be overstated.
ChatGPT felt like a toy; having the same AI power applied to a constantly-updated search database changes the game.
Microsoft was kind enough to provide me with complete access to the new AI "copilot" in Bing. It only takes a few minutes of real-world use to understand why Microsoft (and seemingly every other tech company) is excited about AI. Asking the new Bing open-ended questions about planning a vacation, setting up a week of meal plans, or starting research into buying a new TV and having the AI guide you to something useful, is powerful. Anytime you have a question that would normally require pulling information from multiple sources, you'll immediately streamline the process and save time using the new Bing.
Let AI do the work for you
Not everyone wants to show up to Google or Bing ready to roll up their sleeves and get into a multi-hour research session with lots of open tabs, bookmarks, and copious amounts of reading. Sometimes you just want to explore a bit, and have the information delivered to you -- AI handles that beautifully. Ask one multifaceted question and it pulls the information from across the internet, aggregates it, and serves it to you in one text box. If it's not quite right, you can ask follow-up questions contextually and have it generate more finely-tuned results.

Computing

Forget Dall-E, you can sign up to create AI-generated videos now

A frame from an AI-generated video in claymation style.

Dall-E, ChatGPT, and other AI-generation technologies continue to amaze us. Still, AI image-generation tools like Midjourney might seem boring once you see the new, AI-powered video-generation abilities that will soon be available to us all.

Runway provides an advanced online video editor that offers many of the same features as a desktop app. The company has distinguished its service from others, however, by pioneering the use of AI tools that help with various time-consuming video chores, such as masking out the background.