Anthropic identifies limitations in current AI safety testing methods

Server blades are arranged in a rack within a data centre where artificial intelligence models are processed. Photo: panumas nikhomkhai, Pexels

The Chronify

Share:

By Technology Desk

Anthropic has released a research paper identifying significant gaps in current AI safety evaluation protocols. The company advocates for more rigorous, multi-turn testing as major firms like OpenAI and Google DeepMind continue to develop their own internal safety frameworks.

Anthropic published a research paper on 13 November highlighting critical gaps in existing AI safety evaluation methods. The report suggests that current testing protocols, which often rely on single-prompt assessments and refusal rates, may fail to capture how artificial intelligence models behave in real-world scenarios as conversations develop.

The company argues that as AI systems become more capable of executing multi-step tasks, using external tools, writing code and retaining context across longer interactions, risks may not be apparent in isolated queries. A model might provide a safe initial response but reveal harmful intent or provide dangerous information after a user gradually reframes a request or alters the context of a conversation.

To address these vulnerabilities, Anthropic recommends moving beyond fixed lists of questions. The firm suggests implementing multi-turn simulations where one AI model acts as a user to test the responses of another. This approach aims to identify weaknesses that remain hidden during standard, static safety checks, allowing conversations to develop differently across repeated tests.

Other major technology firms are also refining their internal safety frameworks to address these emerging challenges. OpenAI utilises a Preparedness Framework that integrates automated testing with expert reviews. In an October 2026 publication, the company detailed how it tests monitoring systems against both recorded and simulated incidents, including attempts by advanced AI agents to evade or disable those systems.

Google DeepMind employs a combination of automated assessments and human red-teaming to identify potential risks within its Gemini models. These methods involve specialist teams deliberately attempting to bypass safeguards through role-playing, misleading instructions and repeated questioning to ensure the models remain secure before and after release.

A significant challenge in these evaluations is the increasing reliance on automated systems, often referred to as 'LLM judges', to assess model outputs. While these tools can process large volumes of data, Anthropic notes that they may exhibit biases, favour longer responses or fail to align with human judgement. The company warns that automated scores may not always reflect whether a response is truly safe.

In August 2026, Anthropic introduced the AI Safety Taste Evaluation (TASTE) benchmark to measure how well AI models evaluate safety research proposals. The TASTE benchmark demonstrated that the tested model agreed with experienced human researchers in 60 per cent of cases. This highlights the ongoing effort to ensure that automated evaluation tools remain reliable and effective as the industry seeks to establish common safety standards and greater oversight.

Anthropic’s recommendations for the industry cover the design, execution and reporting of safety evaluations. The company suggests that developers should clearly define the behaviour being tested and distinguish between observational tests, which examine how a model behaves, and causal tests, which assess whether a specific intervention has successfully altered that behaviour.

Furthermore, the company advocates for involving a diverse range of subject matter experts, including psychologists, biologists and political scientists, to help identify realistic risk scenarios. It also recommends running evaluations in controlled, restricted environments, particularly when testing models in high-risk areas such as cybersecurity, to prevent unintended actions outside the test setup.

The industry remains focused on balancing the need for safety with the utility of these models. Anthropic emphasises that evaluations should assess both harmful compliance and unnecessary refusals, ensuring that a model can provide helpful information for legitimate requests while withholding specific instructions that could facilitate misuse. As these companies continue to refine their internal frameworks, the move toward more realistic, multi-turn testing represents a significant shift in how AI safety is measured and maintained.

Elected News

Top Read News