While creating 3D models, editing, compositing and colour grading are much easier and less expensive now, the process of inserting a CG character in a scene is still very complicated. You first create a model with textures and so on, then mess around with motion capture studio green screen. Then those motion primitives need to be applied to the CG skeleton, and the character substituted for the actor. From there, the 3D model needs to match the direction and colour of the lighting, the cast and grain of the film.
It’s complicated stuff that involves a lot of work and generally requires a small to modest-sized team and lots of time to get right. As such, at the top end, VFX with CG characters can cost as much as $20,000 a second to produce.
Wonder Studio is “a visual effects studio in your browser powered by AI” that is as easy to use a Photoshop and enables artists to automatically animate, light and compose CG characters they have dragged and dropped into a live-action scene. All a user needs is a camera, according to makers Wonder Dynamics. You can sign up to a closed beta programme here.
“We built something that automates this whole process, animates it live, frame by frame, there’s no need for mocap. It automatically detects actors based on a single camera. It does camera motion, lighting, color, replaces the actor fully with CG.
You get mocap, clean plate, masks, blender scene, it analyzes the noise and grain. You get your FBX [a common 3D media format] and your textures and it’s assembled on the platform, just like you would import in Blender or Unreal or whatever else.”
Nikola Todorovic
Wonder Dynamics’ co-founders Nikola Todorovic and Tye Sheridan, who starred in Steven Speilberg’s Ready Player One, have longed for a full-scale tool like this that has the ambition of making digital artists’ jobs considerably easier than they are currently. The pair raised a $2.5 million seed round in early 2021 and an additional $10 million A round later that year.
“You 1can’t replace artists with AI, we’re about enhancing and empowering them. This doesn’t disrupt what they’re doing; it automates 80-90% of the objective VFX work and leaves them with the subjective work. The beauty of AI is taking something so complicated and simplifying it.”
Nikola Todorovic
Part of their immediate aim with Wonder Studio is to handle dull, repetitive “objective non-creative” tasks, the technical necessities, leaving artists to be subjective, expressive. Animation and motion data (including hands and face), the shot without the actor or replacement, outlines of characters and objects frame by frame, a 3D representation of the environment with terrain and other features are all included. Everything is matched automatically to the qualities of the shot, or shots, as it can track actors across a full scene with multiple angles.
If you are looking for advice, guidance and support with streamlining your CG character pipelines, we are here to help. Contact John Rowe (john@mondatum.com) or Colin Birch (colin@mondatum.com) for an initial chat.
Source: TechCrunch