Anthropic gives a METR team "employee-like" access to its systems to assess the risks of its models
IA en un minuto newsroom · Editor: Jon Elgezabal
In 30 seconds
A six-person team from the nonprofit METR has settled in inside Anthropic with access almost equal to that of the staff who measure each model's risk there. METR will be able to make its conclusions public without the company blocking them; only a few agreed types of data can be struck out. Jack Clark, co-founder of Anthropic, urged the industry on Friday in Bangkok to pool its brakes while there are no shared rules.
Anthropic announced on Thursday, October 15, that six people from METR, a nonprofit organization that evaluates the most advanced AI models, are now working with company computers, space in its offices and access to internal tools and systems roughly comparable to that of its own risk assessment teams. The company points out that last month it pledged to strengthen independent assessment of its models and its safety practices, and that it signed the White House Accord on Super Intelligence, under which companies promised to work with outside auditors or evaluators. Over the past year, METR had already reviewed several of its risk reports.
Anthropic staff can share information with that team as they would with a colleague, except in a few clearly bounded categories, such as customer data, information covered by attorney-client privilege or the architecture of its models. METR will publish its analysis of Anthropic's risk levels, incidents or practices. The company cannot veto that publication, although it can demand the removal of data in those categories. METR will be able to say publicly whether it thinks those cuts were justified, as long as it does not reveal what was removed. The rest of METR's staff, including those who collaborate with Anthropic on other projects, is kept apart from that team, which operates under enforceable agreements.
Anthropic is preparing three levels of access for outside evaluators. In the first, the evaluator carries out very narrow risk investigations, the company decides what to show it and it can only publish with its permission. In the second, it receives more information and publishes without needing approval, although Anthropic can ask for certain categories of data to be taken out. The third, the broadest, is METR's. The company likens it to bank supervision, in which regulators send examiners to work inside the largest banks.
With other organizations the relationship is at an early stage, with less access and fewer publication rights. SecureBio is reviewing the chapter on chemical and biological weapons in the August risk report, after completing and publishing a pilot review of the previous report; Anthropic updated the announcement that same Thursday to clarify that point. With Accenture and Faculty it is working towards a collaboration to evaluate its models, put them through attack testing, review their alignment and test their safeguards. It is also in initial conversations with other organizations, among them Hugging Face. The company says it is doing this voluntarily and that every company developing frontier AI (the most advanced models) should do the same.
This Friday, at the annual meetings of the International Monetary Fund and the World Bank in Bangkok, Anthropic co-founder Jack Clark called for the industry to develop "braking technologies" and share them while there is no common regulation, according to the EFE news agency in a report published by ABC and La Voz de Galicia. On a panel about the risks and opportunities of AI, Clark argued for common rules and added that "it cannot be only the companies that make these decisions."
Why it matters · analysis and opinion
Until now, almost everything known about the risks of a frontier model came from its own maker, and outside evaluations were limited to testing the product from the outside. Seating an independent evaluator inside the house, with freedom to publish, changes that relationship and looks more like bank supervision than a one-off audit. The real test will come with METR's first reports: how much gets redacted, whether it disputes that in public and whether it says anything uncomfortable for Anthropic. It also remains to be seen whether other frontier AI companies accept something similar, because while this is voluntary and limited to one company, comparing providers stays difficult. Anyone choosing a model for their business will at least have one more independent source to read before deciding.
Official source: Anthropic · Written with the help of AI: how we make the news
