This New AI Tool Wants to Work With Filmmakers—Not Replace Them

The relationship between Hollywood and the AI industry is extremely fraught. Many filmmakers and industry workers abhor the idea of using AI in the moviemaking process, and worry that it will be used as a pale replacement for human creativity. In June, Disney and Universal sued the AI company Midjourney for copyright infringement.  

But several AI companies are nevertheless attempting to forge a path forward in Hollywood. And on Tuesday, one such startup, Moonvalley, took a major step forward by releasing a fully-licensed, professional-grade video model to the public. 

Moonvalley was founded by DeepMind engineers and has close ties with the film industry—the company owns AI film studio Asteria Film Co., which was founded this year by filmmaker and actress Natasha Lyonne and her boyfriend Bryn Mooser. Asteria has been advising Moonvalley on the development of its AI model Marey, which is now available to filmmakers for subscription tiers of $14.99, $34.99, and $149.99 a month.

Marey may become AI’s main entry point into Hollywood, as it’s being developed with the approval of filmmakers and trained on licensed data, theoretically allowing studios to avoid the ethical issues and copyright lawsuits that have plagued the AI industry. 

“We have to make sure that we’re building these tools the right way: building with the filmmaker and the artist at the center of it, rather than trying to automate their job away,” Naeem Talukdar, Moonvalley’s CEO and co-founder, tells TIME.

Moonvalley has raised over $100 million from investors, including Khosla Ventures and Bessemer Venture Partners. Asteria is using Marey for a new documentary about Carl Sagan, to restore and tweak footage. Talukdar also says that Marey is being tested in pilot programs at over a dozen “large studios,” as well as by major advertising companies.

Many other AI video models are black-box systems: you type in a prompt, and it generates a scene wholesale. If you try to tweak one variable in the scene, another may change, making it hard to maintain control of everything that’s been filmed. 

Moonvalley aims to build tools that integrate into the filmmaking process, much like CGI and special effects programs did in the past. Marey allows filmmakers to input storyboards or frames and then tweak them as they see fit, hypothetically giving filmmakers far more control of every detail, from objects to characters to motion to scene composition. 

“It’s this iterative process where you start with some input guidance and then you build up towards the scene that you want, which really isn’t very different from how VFX workflows are today,” Talukdar says. “If you’re an independent studio that doesn’t necessarily have massive infrastructure, you can now, even in a small space, create and curate these scenes in a very granular way.” 

Talukdar walked me through a demo of Marey, first showing actual video footage of an actress in a studio as she turned her head and calmly pulled a gun from beneath her scarf. Talukdar then ran that footage through the model to create a scene of a separate AI-generated woman on a train, who moved the same way as the original actress. He then changed the camera view, swiveling to her other side, and made mountains appear outside the train window. Each generated scene will cost creators roughly $1 to $2 to render, which is comparable to the cost of other AI video generators, and much cheaper than having to physically reshoot footage. 

All of the footage that the model is trained on is licensed from IP owners. About 80% of that footage, Talukdar says, comes from a group of independent filmmakers and agencies that have amassed B-roll over the years. This approach means Marey is trained on roughly one-fifth the data as its competitors, like Google’s Veo 3, Talukdar says. But he claims that Moonvalley is overcoming this deficiency with better technology, created by alumni from Deepmind, Meta, and other top labs. 

“The reality is if we scraped [data], our model would be more powerful, without a doubt,” he says. “But our inclination is that you don’t necessarily have to be the number one model—you just need to be among the best. And I think this is the first generative, fully-licensed model, where you don’t have to compromise quality.”

There are many filmmakers in Hollywood who view AI as antithetical to their creative process. This tension played a major role during the Hollywood strikes in 2023, with many on the picket lines expressing fears about job loss via automation. Talukdar, conversely, argues that AI tools will actually create new types of jobs, and enable studios to push their budgets further rather than slashing them. 

“There’s this idea that instead of spending $50 million on a movie, you can now do it for $5 million, and there’s some truth in that,” he says. “But the other way to think about it—which is how every studio that we talked to is thinking about it—is now for that $50 million and for the same 100 people on that project, they’re just going to be able to do what would have cost them $100 million before,” he says. “It’ll be the same number of people, doing more and better content.” 

Marey’s supporters in the film industry include Ángel Manuel Soto, the director of Blue Beetle and other films. “I feel like Moonvalley and Asteria heard artists’ concerns about ethical AI, and what they created with Marey is a breakthrough,” he wrote in an email to TIME. “From streamlining studio workflows to empowering emerging creators in places like Puerto Rico and Dakar, Marey is the first generative AI that actually gets what we need: a way to move fast, responsibly do more with less, and still protect the people who make this industry human.”

But many filmmakers have cast skepticism on this logic, and fear that they are being set up for a bait-and-switch. “When you look at the larger applications of these technologies, companies and studios never want to use it to empower artists to make cooler stuff for the same amount of money,” Raphael Bob-Waksberg, the creator and showrunner of BoJack Horseman, told Brookings last year. “They want to make things cheaper, cut the artists out, pay people less, and use these technologies in a way that doesn’t make the work better.”