ADL Report Reveals High Failure Rates for Generative AI Video Apps to Block Antisemitic Prompts

The Anti-Defamation League (ADL) published new research on Friday from its Center on Technology and Society (CTS) investigating outputs from artificial intelligence-powered video generating programs. Analysts found that “at least 40 percent of the time” the programs would generate responses when presented “antisemitic, extremist or otherwise hateful text prompts.”

ADL researched fed certain antisemitic prompts into AI chatbot apps — OpenAI’s Sora 2, Google’s Veo 3, and Hedra’s Character-3 model — to gauge how the programs would respond to racist material.

Researchers described how of all four tools, OpenAI’s Sora 2 “performed the best in terms of content moderation, refusing to generate 60% of the prompts.”

The ADL report stated that out of the 50 problematic prompts “Veo 3 only refused to generate ten, Hedra only refused two and Sora 1 refused none. Sora 2 performed the best of all the tools, refusing to generate 30 prompts.”

ADL CEO Jonathan Greenblatt wrote Friday on X about the threat of antisemitic AI videos that, “throughout history, bad actors have exploited new technologies to create antisemitic, extremist and hateful content – that’s where we find ourselves today as AI video generation becomes more sophisticated and accessible.”

A Growing Threat

West Point’s Combating Terrorism Center has previously warned that terrorist groups have started deploying artificial intelligence tools in their propaganda. A January 2024 paper identified the threat which the ADL’s research has now confirmed, with its abstract stating “specifically, the authors investigated the potential implications of commands that can be input into these systems that effectively ‘jailbreak’ the model, allowing it to remove many of its standards and policies that prevent the base model from providing extremist, illegal, or unethical content.”

The unique challenge presented by large language models which experts identify is that responses generated often do not follow the programmed rules, allowing users to generate dangerous content, even when rules are put into place to attempt to prevent it. The creators can put in safeguards against their tools’ misuse, but they so far do not know how to apply them consistently.

Notably, the ADL said that several it shared with OpenAI following initial testing of Sora earlier this year “were refused in the updated version of the tool.”

This new ADL research into generative AI video apps builds on the organization’s findings released in March into AI chatbot apps, when researchers reported varying levels of anti-Israel and anti-Jewish answers from evaluating Chat-GPT (OpenAI), Claude (Anthropic), Gemini (Google), and Llama (Meta).

The ADL called for three policy changes including heavily funding moderation teams, aggressively testing prompts involving hateful stereotypes, and updating keywords in response to real time changes in bigoted vernacular.

“Many of the terms and keywords that are used in hateful or extremist content are obscure and may not be known to engineers or safety practitioners,” the report says. “Video generation platforms should hire research scientists, or other experts within trust and safety teams, who are equipped to keep abreast of the ever-evolving world of extremist rhetoric, which ideally includes partnering with civil society groups who can provide timely analysis and updated language.”

Following the initial Monday report into Sora 2 antisemitic content, The Algemeiner began a further review into material created, hosted, and shared on the app. The videos described here could be found through searching for the terms “rabbi” and “Jews” on the app.

Antisemitism in Action: How Users Navigate Around Safety Features

One way to evaluate how new Sora users are using this technology is through looking at a sequence of videos rather than just one in isolation.

A person with the handle “acm156741” chose to create a series of seven short videos all featuring mixed martial arts star Jake Paul, who allows his likeness to be used in AI videos since he is a self-described “proud OpenAI investor.” In the surreal, comedic videos, the AI-generated Paul uses the claim that he was “promised” something “3000 years ago” as justification for stealing a car and taking candy from a child. Then videos identify him as Jewish, a rabbi, heading for Israel, and money-obsessed.

This technique of identifying Paul as Jewish in one video and then in another video associating Paul with an antisemitic stereotype then overcomes the program’s safety features.