No, De-Aged Harrison Ford Wasn’t All A.I. in ‘Indiana Jones and the Dial of Destiny’

There was no magic tech bullet for crafting Ford's younger face, so ILM assembled a suite of tools that they strategically used on a shot-by-shot basis.

Yes, the de-aging of Harrison Ford in “Indiana Jones and the Dial of Destiny” is an impressive tech breakthrough by ILM that far exceeds the much-debated work on “The Irishman.” No, they didn’t rely solely on A.I. or machine learning. Instead, the de-aging for the Nazi-fighting prologue set in 1944 (with Ford looking 35 years younger) was achieved with every tool in their VFX arsenal — known collectively as “ILM FaceSwap” — and by more than 100 artists.

Was it necessary? Yes, according to director James Mangold, who told IndieWire’s Anne Thompson: “I thought there would be a way to carry the audience through Harrison’s present-day age by greeting them first with an energetic homage to the previous films. You confront the audience with a young Indy in full flower and full action, and then make a sharp turn. Take everything away and let the audience confront the difference between now and then in a brazen way. That was the cut I was most excited about for the movie.”

The prologue, with the Nazis losing the war and hoarding their plunder, also introduced the audience to baddie Voller (Mads Mikkelsen, de-aged as well) and the discovery of Archimedes’ Antikythera (the titular Dial), which carries over into 1969 when the search for the MacGuffin resumes. Ford was also de-aged for a flashback concerning a fight over the Antikythera in 1954.

The ambitious project was overseen by ILM’s Andrew Whitehurst, production VFX supervisor and London-based creative director, and VFX supervisor Robert Weaver. They benefited from Ford’s on-set performance (he began production at the age of 78), as well as the availability of hundreds of hours of footage from the first three “Indiana Jones” movies. This allowed them to capture the actor in every possible way (especially lighting conditions).

“FaceSwap is an umbrella term for a variety of different technologies,” Whitehurst told IndieWire. “We knew intuitively when we started this project — but it proved to be true very quickly — that there was not going to be one technological magic bullet that was going to work for every single shot.

“So we knew we were gonna use a lot of reference from earlier films,” Whitehurst continued. “We knew we were gonna want to shoot with multiple cameras, we knew we were gonna have to build a fully realized CG 3D head, use some machine learning, use paint techniques, use some keyframe animation, use lighting effects, everything. And FaceSwap is the technology to take one face and replace it with another face, whether it’s someone completely different or somebody at a different age.”

The key was building a suite of tools so that the artists could select the correct balance of components to get the desired result for each shot. “I think that was the big breakthrough for us from a conceptual standpoint,” Whitehurst added. “And I think we probably surprised ourselves on a few occasions with what worked on which shots and why.”

The hot-button issue of A.I. (which is a topic of concern in the ongoing SAG-AFTRA and WGA negotiations and is the driving force behind IATSE’s Core Principles on the topic) played a useful role via machine learning. The proprietary program compiled and analyzed all of the existing reference material in 2D, including how the age-specific face should perform. “This repository from the franchise was just a treasure trove,” Weaver told IndieWire, “from the aspect of direct reference as to the look and feel and believability of what we needed to achieve, as well as being able to feed that imagery into a machine learning process.”

But the driving force was Ford’s performance, fed through an improved version of the light-based capture system used on “The Irishman” called FLUX, only with added markers on his face. “He is the character and he’s in terrific shape,” added Weaver, “and so having that and then being able to refer back to the previous films visually, being able to meld those two together, that got us where we needed to be.”

Lighting was essential. As with “The Irishman,” the main unit cameras had additional infrared cameras on either side to capture extra information to help ILM build a 3D version of the face based on the photography, which was useful for then extracting how the light was hitting that face.

“We also shot lighting reference for every single setup with high dynamic range cameras,” Whitehurst said. “We scanned every single set so that we knew exactly where everything was, including the lights. But oftentimes what you need comes down to using all of this reference, but then it goes through artists to use their skill and judgment to just flag this light off a little bit because it’ll add this better bit of modeling to the face or whatever it is.”

Mangold was so pleased with the early results that he added extra action beats to the opening, which begins at night and ends at dawn and revolves around Indy escaping from the exploding castle and fighting off baddies on a motorcycle, in a car chase, and on top of a moving train.” So we often found ourselves, frame by frame, going back to old footage to see exactly the idiosyncratic nature of Harrison’s particular performances,” Whitehurst said. “And the wry smile that he has and the way that the face moves.”

The refining process of every single shot was referred to as under painting. “The subtlety of likeness and performance can’t be tied to one aspect,” Whitehurst said. “You can’t say it’s just the physical position of a face in a moment. A lot of it comes down to how it’s lit. If you light someone from different angles, you read the performance in a completely different way. We were able to generate very quickly low-resolution versions of a face to editorial.

“So when they were cutting, they were never cutting with exactly what we’d shot on the day,” Whitehurst added. “There was a window around it, but there was the beginnings of a period-appropriate performance happening. And that meant that we got, I think, better notes back from the cuts about performance and how it was all working together. And that gave us a big step when we were working to make it better, adding fidelity and refinements to that performance.”

The latest advances in animation allowed them to reproduce such fine details as pores and capillaries while “painting those maps that then make up the holistic look of our hero Indy,” Weaver said. “And there’s subsurface scattering that you dial in just right in areas. It’s never an approach where you just say you need to apply subsurface scattering. It’s more like, it’s working here, it’s not working quite here, and refining those to a very specific set that you’re looking for.

“It would’ve been nice if we had a single recipe that we knew that we had gotten for one shot and now it’s gonna be the holy grail for the rest of the shot work,” Whitehurst added, “but that’s not the case. Every shot is so individualistic and required different artistic feedback and components to manipulate into the final performance.”

.