The universe belongs to Marvel.

With the movie spectacle of the decade running in theatres all over the world, it is not wrong to say that Avengers Endgame, the last movie in a decade long journey of a shared cinematic universe has surpassed all expectations.

From some characters like Captain America, Iron Man and Black Widow making their final appearances in the movie, the scintillating reviews from both audiences and movie critics alike have increased the potential of making it Hollywood’s highest grossing film ever, an accolade which presently lies with James Cameron’s Avatar.

While there is no denying that Marvel Studios has been supremely successful in the execution of a cinematic universe, the absence of villains that could be a real threat to the Avengers was a point where the makers could not cut through successfully. Until Thanos.

Other than Loki, played by Tom Hiddleston and Killmonger, portrayed by Michael Jordon, no single antagonist could hit the hearts of fans with as much impact expected. Not a proper recognition through the span of 22 movies.

Thanos, the purple-faced alien nemesis, made up for all of them with a brilliant screen presence, thanks to the fantastic Josh Brolin who blew life into the character both in Avengers Infinity War and now in Endgame too.

But was it just Brolin that made Thanos the perfect nemesis to the Avengers? No. Marvel Studios have Machine Learning to thank.

Apart from a revolution in CGI, the fact that Thanos was able to display perfect emotions on screen was what made him a force to reckon. Through Thanos, the stakes were high not only in the storyline of both the movies, but they were also huge for the makers, as they had the gap of an excellent villain to fill.

It was important to put emotions on a CGI character’s face to make him resonate more with the audience. This involved portraying the recognizable expressions of Josh Brolin on the Mad Titan’s face.

To achieve this, Digital Domain, one of the digital effect firms for the movie, used a sophisticated machine learning software named Masquerade to make the performance of motion capture more realistic and natural.

The entire process started by correctly putting a hundred to hundred and fifty track dots on Josh Brolin’s face, to be captured by a couple of vertical orientation enabled high-definition cameras.

The scan wasn’t required to provide high-quality results, but a pretty generic render of low quality. This initial rendering then was fed as input to the machine learning algorithm that used from many high- resolution facial scans by a vast variety of expressions.

The Masquerade software opts for those low resolution renders and automatically figures out the high-resolution face shape to be the perfect solution for the screen. If the answer did not seem accurate enough, the team would then tweak things a bit to arrive at a better solution.

These tweaks involved instances like raising the brows higher or a little bit of lip compression, which went back into the system and were then learned by the machine learning algorithm.

Subsequently, further results through the low mesh came out better, but all of this was just a single step. The next step in the process is known as direct drive, which plucked the high- res face mask function to place it on the villain’s character model.

If there were no machine learning system like Masquerade in place, the Visual Effects team would require to change the expressions manually through animation, where the results were surely not to be as impressive like the ones coming with the help of Masquerade. It would have been a time- consuming process too.

However, there are also other advanced techniques like FACETS, used for facial tracking in Avatar and even the Planet of the Apes trilogy.

It is quite clear that if you are not using machine learning in your software to enable better CGI and VFX, you are never going to get the final outputs as you expect them to be. In the times ahead, technology will be used more for things more than faces.

To cut a long story short, expect machine learning to have an integral role just about anywhere when it comes to special effects and design.

To get the best machine learning systems/solutions for your own business or company, let us help you with the best in class recommendations & solutions.