Dall-E, ChatGPT, and other AI-generation technologies continue to amaze us. Still, AI image-generation tools like Midjourney might seem boring once you see the new, AI-powered video-generation abilities that will soon be available to us all.
Runway provides an advanced online video editor that offers many of the same features as a desktop app. The company has distinguished its service from others, however, by pioneering the use of AI tools that help with various time-consuming video chores, such as masking out the background.
Now Runway has pushed its video magic to a new level, announcing on Twitter a new AI video-generation offering that exceeds anything that’s been shown before, even by leading AI companies like Google and Meta. Known as Gen-1, this AI model uses a video that you supply, along with a prompt, to generate a remarkably stable and high-quality rendering that matches the structure of the original, but with a very different style.
Runway is a new face in the world of AI-generated content, but it comes from the co-creators of Stable Diffusion, which is a popular text-to-image AI tool available on Mac.
The prompt can be text-based, or you can supply an image to use as a style to process the video. There are five modes of operation currently. Stylization mode captures the look of an image and applies that to the video. Storyboard mode replaces more of the original video. The idea is that you are only supplying placeholders for the AI to replace.
For more precision, Mask mode lets you specify which portion or subject in the video should be affected by the AI process. Render mode is similar to Storyboard, adding plenty of extra details to the supplied video. Runway also has a Customization mode that provides many more options to dial in what you want.
Even in this nascent form, it’s far better than videos that have been painstakingly assembled from hundreds of AI-generated images. Instead of a chaotic roiling, the sample videos are cohesive and flow much more naturally.
Early access to Gen-1 started today. And the results are already blowing us away. h/t @MatthewPaquette
Join the waitlist today. Share with friends to get in sooner. #Gen1 https://t.co/a41n97xl1u https://t.co/eWHLkeMqWx pic.twitter.com/U2nF3z3DZ7
— Runway (@runwayml) February 8, 2023
Runway’s Gen-1 video generation is already available in early access, and the tweet above details how you can join the waiting list. In the meantime, check out the YouTube video from Runway, which has better resolution than the Twitter video. The good part is about 40 seconds in.
