Runway adds startling Gen-1 text- and image-to-video generator

Working much better than other, similar AI-powered tools that generate videos assembled from hundreds of (AI-generated) images, Runway, from the makers of text-to-image AI tool Stable Diffusion, is a feature-rich video editor that separates itself from the herd.

With 30+ AI Magic Tools, real-time video editing, collaboration, and more, Runway is your next-generation content creation suite. Right inside your browser

A new AI video generation tool. Gen-1, uses a machine learning model to take a video that you supply plus a prompt and generates a new, high-quality rendered output that matches the structure of the original, but in a different style.

Prompts can be text-based or an image can be used as a style to process the video. There are five modes of operation currently:

  • Stylization mode captures the look of an image and applies that to the video
  • Storyboard mode replaces more of the original video – you are only supplying placeholders for the AI to replace
  • Mask mode lets you specify which portion or subject in the video should be affected by the AI process
  • Render mode is similar to Storyboard, adding plenty of extra details to the supplied video
  • Customization mode that provides many more options to dial in what you want.

“The Simpsons” but make it an experimental cubist stop motion. Felt right given long tradition of reanimating the Simpsons intro. Created with the spellbindingly addictive #Gen1 AI video generator from @runwayml — still early days but step in the future of #animation #ai #aiart #simpsons #experimental @The Simpsons

♬ original sound – Paul Trillo

Here’s what it does with the intro to The Simpsons re-worked as experimental cubist stop motion.

Sources: digitaltrends