Back

Runway adds startling Gen-1 text- and image-to-video generator


Working much better than other, similar AI-powered tools that generate videos assembled from hundreds of (AI-generated) images, Runway, from the makers of text-to-image AI tool Stable Diffusion, is a feature-rich video editor that separates itself from the herd.

With 30+ AI Magic Tools, real-time video editing, collaboration, and more, Runway is your next-generation content creation suite. Right inside your browser

A new AI video generation tool. Gen-1, uses a machine learning model to take a video that you supply plus a prompt and generates a new, high-quality rendered output that matches the structure of the original, but in a different style.

Prompts can be text-based or an image can be used as a style to process the video. There are five modes of operation currently:

  • Stylization mode captures the look of an image and applies that to the video
  • Storyboard mode replaces more of the original video – you are only supplying placeholders for the AI to replace
  • Mask mode lets you specify which portion or subject in the video should be affected by the AI process
  • Render mode is similar to Storyboard, adding plenty of extra details to the supplied video
  • Customization mode that provides many more options to dial in what you want.
@paultrillo

“The Simpsons” but make it an experimental cubist stop motion. Felt right given long tradition of reanimating the Simpsons intro. Created with the spellbindingly addictive #Gen1 AI video generator from @runwayml — still early days but step in the future of #animation #ai #aiart #simpsons #experimental @The Simpsons

♬ original sound – Paul Trillo

Here’s what it does with the intro to The Simpsons re-worked as experimental cubist stop motion.

Sources: digitaltrends



RELATED INSIGHTS