Anthropic gives six METR evaluators employee-like access
METR can publish its risk findings on Anthropic without the company's approval. Our four-test check, built from Anthropic's list, works on any AI lab's safety claims.
If your team runs on Claude, METR says it will publish its own read on Anthropic's risk posture, and Anthropic says it "can't veto publication," only require narrow redactions. Bookmark METR's risk assessment page and use the four-test check below on any AI vendor's safety claims.
What changed#
On Oct 15, Anthropic said it has onboarded six METR employees "with company hardware, office space, and access to internal tools and systems that are roughly on par with what our own risk assessment teams have access to." METR is a nonprofit that evaluates frontier AI models.
Before: METR reviewed Anthropic's risk reports from the outside, and on Sep 18 Anthropic said it was only "in dialogue with METR" about piloting embedded evaluation. After: METR staff work inside, and "Anthropic can't veto publication." Anthropic can require redactions only in narrow, predefined categories, and METR may say publicly "whether it believes the redactions were justified, as long as it does so without revealing the redacted information."
Anthropic also named two other evaluators. SecureBio is reviewing the chemical and biological weapons chapter of the August Risk Report. Accenture, with its AI unit Faculty, is working toward evaluating and red-teaming models. Talks with Hugging Face and others are early.
Why it matters#
Anthropic says some safety information can't go public because it could help attackers or is commercially sensitive. Its fix is to let outsiders see it and publish their conclusions instead. Anthropic compares this to bank examiners working inside large banks, while saying the analogy "isn't exact." It says "every frontier AI company should do the same." According to Reuters, Anthropic and OpenAI had earlier sought significant control over what outside reviewers could publish.
How to use it#
Anthropic lists four things an evaluator needs. Our checklist turns them into a test for any lab's safety claim:
- Expertise in frontier model evaluation, including how tests can fail.
- Independence: no equity in the company, no financial or operational dependence on it.
- Access to models, training details and how safeguards work.
- Freedom to publish findings.
Prompt (ours):
Here is an AI company's announcement about outside safety evaluators: [paste]. For each of these four tests, say whether it is met, partly met, or not met, and quote the line that shows it: the evaluator's expertise, independence (equity, funding, operational ties), access (models, training details, safeguards), and freedom to publish without the company's approval. Then list what is excluded from the evaluator's view.
Good to know#
- Three tiers. Tier 1 evaluators run narrow investigations and publish only with Anthropic's approval (SecureBio is here). Tier 2 can publish without approval. Tier 3 adds employee-like access (only METR so far).
- Still off limits: per Anthropic's examples, customer data, attorney-client privileged information and model architectures. Anthropic says architecture leaks are high risk and usually low relevance, and it will "generally seek to share" them when they matter.
- Siloed team. Other METR staff, including those on other Anthropic projects, are walled off from the embedded team's work.
- Who pays. Anthropic funds Accenture's work directly. Its Sep 18 post said nonprofits like METR would pilot embedded evaluation "using their own funding," and METR says it "does not accept compensation" for this work.
- Watch next: METR's first published assessment. METR told Reuters its report "will openly describe the scope, access, and other limitations of our work." On Oct 14 METR posted four proposals for how an assessor could investigate a lab, covering alignment, AI R&D acceleration, agent control, and informing the public.
Sources#
- https://www.anthropic.com/news/an-update-on-independent-evaluators (checked 2026-10-15 3:51 PM ET)
- https://x.com/AnthropicAI/status/2110784662071398807 (posted 2026-10-15 1:27 PM ET, checked 3:52 PM ET)
- https://www.anthropic.com/news/accenture-embedded-evaluation (checked 3:53 PM ET)
- https://metr.org/ and https://metr.org/blog/2026-10-14-our-approach-to-risk-assessment/ (checked 3:53 PM ET)
- https://www.channelnewsasia.com/business/anthropic-grants-outside-evaluator-metr-employee-level-access-in-ai-safety-push-6463306 (Reuters, published 2026-10-15 1:35 PM ET, checked 3:54 PM ET)
Last checked: 2026-10-15