Google's DeepMind and YouTube built and shelved 'Orca,' a 'mind-blowing' music AI tool that hit a copyright snag

• Google's DeepMind and YouTube previously built and shelved Orca, an AI music tool.
• Orca could generate music mimicking artists. Google trained it on copyrighted YouTube music videos.
• Google's AI strategy led to Orca's development. Legal risks halted its release.

Name your favorite artist, choose a genre, and feed it some lyrics, and AI will create a song that sounds completely authentic.

That was the vision of "Orca," a project Google's DeepMind and YouTube collaborated on and ultimately shelved last year after butting up against copyright issues, according to four people familiar with the matter, who asked to remain anonymous because they were not permitted to talk to the press.

The tool, which was internally codenamed "Orca," let anyone generate music with just a few simple prompts. It was developed as Google scrambled to catch OpenAI.

Users could generate a new song by giving Orca prompts like a specific artist, lyrics, and musical genre, said one person familiar with the project. For example, they could use the tool to generate a hip-hop song with the voice of Taylor Swift, that person said, adding that it was "mind-blowing."

Google eventually approached some music labels about releasing the Orca tool to the public, offering a revenue-share agreement for the music and artists Orca trained from, and the labels demurred, forcing Google to put the brakes on the project, that person said, adding that it was a "huge legal risk."

Orca is yet another example of how tech companies have moved at breakneck speeds to get ahead in the AI race. It also demonstrated how tech companies were willing to ride roughshod over their own rules to compete.

Google had previously avoided using copyrighted videos for AI training. When OpenAI started scraping YouTube for its own models, Google leadership decided to be more aggressive and reneged on its rule, said a person with direct knowledge of Orca.

Google has terms that allow it to scrape data from YouTube videos to improve its own service, although it's unclear if building an AI music generator would fall under this policy.

Developments on Orca throughout 2023 were so promising that at one point, some employees suggested that giving it a codename after a killer whale wasn't a good idea if DeepMind was about to destroy an entire music industry, one person involved recalled.

Some researchers inside Google had developed a similar model of their own, MusicLM, trained on "a large dataset of un-labeled music," as detailed in a paper published early last year.

In November 2023, DeepMind announced a music generation AI model named Lyria, which was a pared-down version of the Orca project. Users could ask Lyria to generate music using the voice and music style of some artists who had explicitly worked with Google on the project, such as John Legend — although it was far more limited in scope than Orca, three people familiar with the project said.

Some employees who worked on Lyria and Orca left to found a new startup named Udio, which makes an AI music creation app.

Google did not respond to a request for comment.

Are you a current or former DeepMind or YouTube employee? Got more insight to share? You can reach the reporter Hugh Langley via the encrypted messaging app Signal (+1 628-228-1836) or email (hlangley@businessinsider.com).