Chinese animation house puts AI capabilities to the test

Chris Bremble, founder of a Beijing-headquartered visual effects and animation firm called Base Media and an Emmy award winner, remembers the first time he heard about generative artificial intelligence.

It was at an industry conference in 2018. He saw a demonstration of software that could replicate smoke from a photograph within five seconds.

“I had a small panic attack,” Bremble said. “Seeing that thing made me go, ‘Yeah it’s coming. We can’t avoid this.’”

Five years later, generative AI has not been widely adopted by the movie post-production sector yet, but a lot of actors and writers in Hollywood are already nervous about its impact on their art and jobs. It is partly why they’re on strike.

In China, where Bremble has offices in a few cities, there are already reports of layoffs in the video gaming industry because of generative AI. Bremble is keen to stay ahead of the curve on the technology.

Base Media has worked on Hollywood blockbusters such as “Black Panther: Wakanda Forever;” the “Star Wars” limited series “Obi-Wan Kenobi;” and Chinese films like “The Wandering Earth.” Earlier this year, the firm was commissioned by Chinese streaming giant iQiyi to look into the potentials of generative AI. Specifically, iQiyi’s children and animation division, Wonderworks, wanted to test how well AI generated 3D locations, which is the most expensive and labor-intensive part of animation production.

“For example, if it’s “Aladdin,” [you] have one castle here, one city corner, and you want the character to run through it, and you want a bird’s eye view of the character meandering in the alley. Then you actually have to [virtually] build the whole thing,” Wang Huiyu, head of Wonderworks, said. “Every storefront needs to be built and every window.”

She said 30% of her animation production usually goes toward creating locations.

To test the capabilities of AI, Wang gave Base Media a 90-second script about a girl looking for her friends and an iQiyi animation series she has just finished called “The Roofus.” All 260 minutes of the animation series were fed into the AI machine for it to replicate the look and style of the animation Wang wanted in the final product.

Base Media’s visual effects supervisor Igor Lodeiro in the Xiamen office led the project. He has won a daytime Emmy and has worked on many Oscar-nominated films. Lodeiro said he had low expectations of the AI tools initially.

His team put together a rough cut. They animated the lead character by hand and put gray blocks in the background and everywhere else. The team input photos and keywords to prompt the AI.

“[It] was pretty freaky at the beginning,” Lodeiro said. “There were some anomalies and distortion, but you look at the [initial] frames [of the test video] … it looked like it could belong to [iQiyi’s “The Roofus”] show. I was surprised.”

In the final video, the animated character skateboards down a wooden bridge in Beijing, runs across curved Chinese rooftops and hangs off a gargoyle overlooking the sprawling city of Paris.

AI generated every location the character passes through; the skateboarding motion was also created by AI. AI tools are not perfect, and some tweaking was done by hand. The final video took three weeks to complete. It would have taken more than eight with a human animation team.

As a viewer, it is hard to discern what was done by AI and what was not.

“That is the best thing we could hear,” Bremble said. “The audience shouldn’t be watching a show and going, ‘Oh, this is AI-generated.’”

He is keen to see how AI can help his top talent do more in a day and thereby make production scheduling more predictable.

While Lodeiro can see that the more tedious parts of animation production — like storyboarding — might be done completely by AI, he thinks eventually audiences might get better quality animations.

“If they let us go crazy with AI, we can create looks that are very hard to achieve on a conventional pipeline. So, we will be able to create very unique things that are not typical,” Lodeiro said.

He is very excited about AI’s potential. And he does not worry about his or his team’s job security.

“[These AI tools are] power tools. [They] let us do more [and] better,” Lodeiro said. “This is a very competitive market. If you now can produce higher quality at the same price, now you have the upper hand.”

But Base Media is proceeding with caution, because not every client is on board with AI.

“Our Chinese clients are saying, ‘AI, AI, AI. How can you do more [with AI]?’” Bremble said. “Our U.S. clients [need us] to sign an agreement [promising that] no AI has been used in the creation of [the] work at all.”

That’s because in the U.S., there is more concern about copyright infringement, he said.

iQiyi Wonderwork’s Wang in China said she is worried about copyright infringement, but is more excited about AI’s ability to boost her creativity in a very competitive environment. Wang wants to see what AI could do in other areas, such as making food look more realistic in animations.

“Real food takes a lot of patience and detail to make the textures, the steam and the transparency,” Wang said. “I haven’t tested with nature, like flower leaves and dew.”

After a few more trials, Wang wants to do a whole movie using generative AI.

Additional research by Charles Zhang.