Three in five AI models fail a terrorism safety test, according to Tech Against Terrorism

Three in five AI models have failed a terrorism safety test, according to research that CBS News reported on Friday, October 9.

The research is by Tech Against Terrorism, a UK-based nonprofit organization that works to disrupt terrorist activity online, which shared it with the network. It tested more than 130 models with hundreds of requests like those someone plotting an attack might make. The organization defines failing as giving one complete, specific answer about a mass-casualty subject, or scoring below 90 out of 100 on its benchmarks, which measure how consistently a model refuses a request, weighted by the severity of the subject.

Open-weight models, whose parameters are public and can be modified by anyone, scored similarly to closed ones. But those that had undergone "abliteration", a process that completely strips away their guardrails and to which open-weight models are vulnerable, failed every time. They are vulnerable because the patterns a model learns in order to identify harmful requests sit in its weights, where they can be found and canceled. Meta's Llama 3.1 8B went from 97 points out of 100 to around 3 in its modified version.

Meta told CBS News that Llama 3.1, introduced in 2024, undergoes safety evaluations and risk assessments, and that its use policy prohibits uses that could be harmful or illegal.

According to Tech Against Terrorism, the process can be done for free with tools available online and, for smaller models, in a matter of minutes. Modified versions of popular open-weight models appear less than three days after their release. As of late September, Hugging Face, which the organization describes as the largest public model repository, hosted more than 29,000 repositories advertising models as uncensored or without safeguards.

Yacine Jernite, head of machine learning and society at Hugging Face, told CBS News that the platform conducts ongoing moderation and acts on content that goes against its policy. In Jernite's view, the report provides useful tools, but some of its recommendations are incompatible with open research, and a model without guardrails should not be equated with a harmful one.

The report clarifies that its benchmark measures whether a model hands over what was asked, not whether a person could act on it. It also says that, aside from one extremist chatbot identified by the group, it found no evidence of these models being used by terrorists or extremist groups.

Tech Against Terrorism, which receives backing from several governments and is supported by the UN Counter-Terrorism Directorate, proposes that governments and developers fund independent benchmarks, that models be made harder to modify this way before release and that public repositories prohibit stripped models. It says it is not asking for a slowdown in AI development or an end to open-weight releases. The organization sent its findings to the companies named in the report on October 8.