A team of Netflix researchers, including Paul Debevec, has created a new, faster and more accurate AI-powered form of green screen – Magenta Green Screen (MGS) – that uses magenta light and can produce realistic VFX in real-time. Actors are filmed against a background of bright green LEDs. They are lit from the front by red and blue LEDs, which together create a magenta glow.
Because digital cameras work by taking an individual red, green and blue value for each pixel, this technique has the effect of creating a green channel that records only the background, with the foreground appearing black, and red and blue channels that record only the foreground, leaving the background looking black. Together these create the magenta and green look. Film editors can replace the green channel in real time, realistically and instantly placing the actors in the foreground of another scene, with even potentially tricky areas, such as transparent bottles or the area around strands of hair, working without problems.
Using machine learning, though not operating in real-time, the full range of colour is put back into the foreground, using a photograph of the actors lit normally as a reference to create a realistic-looking green channel.
The technique is complex and requires the overall system to be extremely accurate so is this a practical solution for all?
Source: New Scientist