What Is AI-Generated VFX?
When you think of VFX, normally you’d think of animated movies and videos, you’ll think of all those strange pyjamas actors need to wear, if they’re filming fantasy or sci-fi films. Traditionally, editors and animators would spend hours, meticulously creating the code and visual effects that make up those animated films. Nowadays, however, animated films and videos seem to be cropping up everywhere, almost on a daily basis. How? Why?
The trick to this is AI-Generated VFX.
What is AI-Generated VFX you say? Well, it’s what it says it is. It’s visual effects that are created and generated with the use of AI technology.
The Technology Behind AI-Generated VFX
How are visual effects being created by AI technology? The key word here is “machine learning”. Animators are using this type of technology to create animations that seem more lifelike and real, despite the fantastical elements. Take the animation in the latest films and TV shows for the Marvel Cinematic Universe. Many of the elements in those films, like Wanda’s superpowers, or Thanos’ height and build are impossible to find in reality. After all, it’s very rare someone is more than 8ft tall, and it’s impossible for Josh Brolin to be that big, or purple, yet on screen, he’s exactly that.
This was done through AI-generated VFX, using machine learning technology. Essentially, the computer, where the animation is generated, is programmed to “learn” what characteristics to look for and apply the animation to those characteristics. So, to make Josh Brolin the 8ft titan he is, the actor had to wear a back brace that mimicked the height of the titan, along with a motion capture suit, for the computer to recognise. This then would make it much easier for the animator to create Thanos, in all his gigantic purple glory and fury.
The Possibility Of Time Travel and De-Ageing
Of course, using AI-Generated VFX isn’t just used for making something impossible possible, it can also mimic time travel. Of course, not literally. When a new fantasy or sci-fi film is released, there are always going to be near impossible odds and concepts, like time travel. Sometimes, the film’s plot includes something like time travel and flashbacks, into eras that are difficult to replicate in reality, especially if a lot of time has passed, particularly for actors.
For example, in Netflix’s original film, The Irishman (2019), AI technology was used to “de-age” the actors in the film, particularly the main characters, like Robert DeNiro, Al Pacino, and Joe Pesci. The film itself is based around the life of Frank Sheeran, who was employed by the Bufalino crime family. The film covers the time from when he was first employed by the family, to the time after, when he’s retired. Throughout the film, Robert DeNiro’s face changes from young to old, and yet, this was filmed in 2019, when DeNiro was 76 years old. It’s highly unlikely that the film began shooting when DeNiro was 35 years old (the age where the real Frank Sheeran worked with the Bufalino family), so technology needed to be used to make DeNiro look younger.
That’s where AI-Generated VFX comes in. To help DeNiro look younger, machine learning technology was used to track his features and mask the actor’s younger self over his real features. Of course, if you watch Behind-The-Scenes footage, DeNiro doesn’t wear those strange pyjamas that many actors normally wear, especially for the bigger, more epic, fantasy and sci-fi films that Hollywood produces. Instead, he’s acting as he normally does, in front of a regular camera. So, instead, the de-ageing process takes place on the editor’s computer, using the technology that the ILM (Industrial Lights & Magic) labs have, since they were a partner in the filming process.
Bringing The Dead Back To Life
Of course, the use of AI-technology in VFX doesn’t literally mean that the dead come back to life. It’s used only as an illusion. Essentially, the use of AI-Generated VFX is best used for superimposing the face of a dead person, onto someone else, who has a similar build, facial features, and mannerisms. For example, in the 7th instalment of the Fast And Furious films (2015), at the very end of the film, as a tribute to Paul Walker, who had died during the shooting period of the film, Dominic Toretto and Brian O’Connor had one last race.
However, because Walker had died prior to these shots, someone had to take his place. Of course, it’s unrealistic to expect the film crew to find someone who looked like Walker, on a moment’s notice. So, instead, they had Walker’s brother play the role, while they used CGI and AI-Generated VFX to superimpose Walker’s face onto his brother’s. This created the illusion that the actor was still alive. Similarly, in the Star Wars films, Carrie Fisher’s face had to be used in some of its more recent instalments, like in Rogue One (2016), where the face of a young Princess Leia had to be superimposed onto the face of the actress who played her.
This Is Just The Start
Of course, being able to de-age someone, or make them seem like a different creature altogether, is just the tip of the iceberg. There are many other uses to AI-generated VFX, like building impossible worlds, making magic or superpowers real, or even performing dangerous stunt sequences, for an action scene, and filmmakers are only just now scratching the surface of AI technology.
Nevertheless, AI-generated technology is beginning to take over the cinematic and filmmaking industries, because of the possibilities it promises. After all, why just imagine a dragon burning down a city with its fire breath, when films can make it possible? Even watching cars transform into robots, or spells being cast from wands, seem natural on a cinema screen. This is all thanks to the use of AI technology and computer animation. So, now you know what AI-generated VFX is, the next time you watch a fantasy, superhero, or sci-fi movie, remember that what you’re seeing wouldn’t be possible without it. After all, it used to take hours, days, weeks, even months, for animators to create just one scene of a film, using traditional CGI software.
