The predictions are that, through tools like Stable Video Diffusion, Pika, Runway, Meta’s Emu Video and, soon-ish, Midjourney, generative video is going to be one of the big talking points and disruptors in 2024
And if the predictions are correct, more video content creators will start to learn how to automatically edit footage, synthesise voiceovers, generate soundscapes and overlay VFX while telling complex stories. Professional workflows and pipelines will start to become more mainstream.
Professionals working in media production, gaming, architectural visualisation, VFX and animation will be able pre visualise, prototype and refine content much faster than before. World builders will start to automatically populate interactive landscapes, building interiors, and even entire metropolitan cityscapes with just text prompts.
Elsewhere, machine learning will speed up 3D character design, motion synthesis, lip-syncing, scene choreography and rendering, and, with the simulation of physics and materials, enable entire animations to be synthesised from scratch.
AI-powered narration, including of audiobooks and podcasts, music generation, acoustics modelling, and mixing/mastering will open up new modes of soundscape creation.
All of this “holistic scene generation” and workflow optimisation is powered by multimodal models that assimilate video, audio, speech, 3D environments, physics and language into a common representational framework.
So, if this is all going to happen as the experts say it is, the sky’s the limit and the creative opportunities are boundless. However, while you’re looking into all of this in the coming months, do always remember, please prompt responsibly.
Mondatum Labs‘ highly-skilled team of software developers, programmers, project managers and media technologists can help you to realise your artistic goals. They use the latest in machine learning-based technology combined with good old mathematical optimisation and the established technologies and techniques you know and trust.
To find out how we can support, guide or advise you on your next project, contact Colin Birch (colin@mondatum.com) or John Rowe (john@mondatum.com).
Source: Futurist Speaker
Musical notation image ℅ Symphony