Red-Teaming AI Systems for Biosecurity Risks

Biosecurity red-teaming is no longer adequately described as a set of adversarial prompts. It is a controlled assurance process that tests the model, the deployed system, the people using it, the safeguards surrounding it, and the governance decisions that follow. A credible evaluation therefore distinguishes what a system can do from whether it increases human or agent capability, whether that uplift changes operational risk, and whether safeguards remain effective over time.

Learning Objectives
  • Distinguish capability measurement, uplift, safeguard effectiveness, operational validation, and governance assurance.
  • Specify a biological misuse threat model before selecting tests or thresholds.
  • Evaluate the full deployed system, including tools, memory, access tiers, monitoring, and account-level controls.
  • Interpret red-team findings without overstating negative results, jailbreak success, or benchmark performance.
  • Design an evidence package that supports a documented deployment decision and subsequent reassessment.

Biosecurity red-teaming should be organized around five analytical layers: capability measurement, human and agent uplift, safeguard effectiveness, operational validation and risk translation, and governance assurance with lifecycle monitoring. Threat-model quality and statistical validity apply across all five.

Key conclusions:

  • Strong benchmark performance does not establish physical-world uplift or severe operational risk.
  • A refusal bypass is not equivalent to useful harmful assistance, and a blocked prompt is not evidence that the broader safeguard stack is adequate.
  • Evaluations should identify the exact model, system configuration, access mode, tools, evaluator resources, and testing date.
  • Results are lower bounds on elicited capability and time-bounded snapshots of safeguard robustness.
  • Red-teaming should end in a residual-risk decision with defined remediation, retesting, monitoring, and incident-response obligations.

Introduction

Red-teaming is a structured effort by authorized evaluators to identify vulnerabilities, limitations, or misuse opportunities in an AI model or system. In biosecurity, the objective is not to generate operationally sensitive content for public release. The objective is to determine whether an AI system changes a defined pathway to severe biological harm, whether deployed controls interrupt that pathway, and whether the resulting evidence supports deployment, restriction, remediation, or deferral.

The International AI Safety Report 2026 describes red-teaming as one part of a broader risk-management system that also includes threat modeling, benchmarks, audits, safeguards, incident reporting, and risk-acceptance decisions. It also cautions that red-team findings depend on evaluator composition, instructions, attack rounds, model access, and tools, and that failure to identify a risk does not establish that the risk is low (International AI Safety Report, 2026).

Scope and Defensive Boundary

The scope is evaluation design, evidence interpretation, safeguard assurance, and governance. Actionable biological procedures, sequences, quantities, acquisition guidance, and detailed methods for bypassing safeguards are excluded. Sensitive test cases should be maintained in controlled evaluation environments with institutional oversight, access logging, and release review.

Public reports should provide enough information to assess validity without reproducing operational details. Appropriate disclosures include the threat model, evaluation classes, baselines, model and safeguard versions, evaluator access, budgets, scoring methods, uncertainty, limitations, and remediation status.

Five Analytical Layers

The term red-teaming is often used for several different forms of evidence. Treating them as interchangeable produces unsupported conclusions.

Layer Primary question Appropriate evidence Invalid inference
Capability measurement What can the model or system do under strong, authorized elicitation? Private benchmarks, expert grading, tool-enabled tasks, longer rollouts, safeguards-disabled testing where authorized High benchmark scores establish real-world misuse
Human and agent uplift Does access improve performance relative to an appropriate baseline? Randomized studies, matched controls, expert and novice strata, agent baselines Better answers establish completion of an operational pathway
Safeguard effectiveness Does the deployed control stack prevent relevant assistance under realistic pressure? Refusal and utility tests, adaptive attacks, account-level testing, access-control review, monitoring evaluation A high refusal rate proves adversarial robustness
Operational validation and risk translation Does measured uplift remove a material bottleneck in a defined pathway to severe harm? Benign proxies, controlled validation, structured expert judgment, pathway analysis A single successful task establishes catastrophic risk
Governance assurance and lifecycle monitoring Are thresholds, decisions, monitoring, remediation, and independent review adequate? Risk reports, decision records, incident exercises, reassessment triggers, external review A third-party label alone establishes independence

Threat-model specification and statistical validity are cross-cutting requirements. A technically sophisticated benchmark is not decision-relevant if it measures the wrong actor, access condition, or bottleneck.

Threat Model Before Test Design

A threat model should be recorded before evaluation tasks and success criteria are finalized. At minimum, it should define:

  • Actor: relevant expertise, resources, institutional access, and intent.
  • Baseline: information, software, equipment, collaborators, and time available without the evaluated system.
  • System access: model version, interface, context window, tools, retrieval, memory, fine-tuning permissions, rate limits, and access duration.
  • Pathway: the high-level sequence of decisions or bottlenecks being evaluated, without publishing operational details.
  • Outcome: the severity threshold and the evidence required to conclude that a bottleneck has been materially reduced.
  • Safeguards: model behavior, classifiers, monitors, identity controls, rate limits, account enforcement, trusted-access arrangements, and weight security.
  • Adversary budget: evaluator expertise, time, queries, retries, compute, collaboration, and permitted adaptation.

The comparison between evaluator resources and the modeled threat actor should be explicit. A red team with less access, less expertise, or less time than the actor of concern can produce only a lower bound on vulnerability. A substantially stronger red team may identify valuable weaknesses, but its findings require translation before they are attributed to a lower-capability actor.

Evaluating the Full System

The unit of analysis should be the deployed system rather than the language model alone. Tools, retrieval, memory, agents, fine-tuning, user identity, and monitoring can materially change both capability and risk. The same model may therefore require separate assurance for public chat, application programming interfaces, trusted-research access, agentic products, and open-weight release.

OpenAI’s July 2026 system card illustrates this system-level approach. OpenAI reported treating all three GPT-5.6 models as High capability for Biological and Chemical risk under its Preparedness Framework, while also stating that its evaluations are lower bounds because different prompting, fine-tuning, scaffolding, interactions, or longer rollouts could elicit additional capability. Its reported safeguard stack includes model-level training, real-time controls, cross-conversation monitoring, actor-level enforcement, trusted access, external evaluation, and continuing red-teaming (OpenAI, July 2026). These are developer-reported assessments, not independent certification.

Anthropic’s July 2026 Claude Opus 5 system card reports treating the model as having CB-1 capabilities but not CB-2 capabilities while applying ASL-3 protections. Anthropic used automated chemical and biological assessments for that release rather than new expert red-teaming or uplift trials. This distinction illustrates why capability classification, safeguard posture, and evaluation type should be reported separately (Anthropic, July 2026). This is also a developer-reported assessment.

The relevant threat model may involve repeated, tailored interactions rather than a single unsafe response. Evaluation should therefore include multi-turn sessions, longitudinal account patterns, tool use, access-tier changes, and recovery after partial refusals. It should also test whether remediation prevents recurrence across materially similar contexts.

Evaluation Design

Evidence Realism

Evaluation methods form a ladder rather than a single hierarchy. Each level answers a different question:

  1. Automated benchmarks: scalable screening of defined knowledge or task classes.
  2. Open-ended expert evaluation: assessment of reasoning quality, completeness, and error severity.
  3. Interactive human or agent studies: measurement of performance relative to a defined baseline.
  4. Benign proxy or controlled physical validation: testing whether in silico performance transfers to execution without creating unacceptable risk.
  5. Post-deployment evidence: monitoring, incidents, near misses, abuse investigations, and remediation performance.

Recent preprints illustrate the difference between constructs. BioTIER contains 542 expert-curated prompts across risk and legitimate-use categories, supporting evaluation of calibrated refusal rather than general biological capability (Marshall et al., 2026, preprint). ABC-Bench evaluates agentic biological tasks and includes limited physical validation, supporting assessment of tool-enabled capability rather than an inference of misuse likelihood or safeguard adequacy (Liu et al., 2026, preprint).

Capability Elicitation

Capability evaluations should make reasonable efforts to elicit the system’s strongest relevant performance. Depending on authorization and risk controls, this can include expert prompting, longer rollouts, tool augmentation, multiple attempts, and testing of configurations with reduced deployment safeguards. The report should distinguish the capability of the underlying model from the behavior of the deployed product.

Every result should specify:

  • exact model and checkpoint or release date;
  • system prompt and policy version where disclosable;
  • enabled tools, retrieval, memory, and scaffolding;
  • inference settings, test-time compute, attempts, and stopping rules;
  • whether safeguards were active, reduced, or bypassed under authorization;
  • evaluator expertise and access;
  • task source, contamination controls, grading method, and uncertainty.

Baselines and Uplift

Uplift is a causal comparison. The appropriate baseline is not automatically “no assistance.” It may be internet search, existing software, a lower-capability model, a domain expert, or the same user without agentic tools. Baseline access, time, incentives, and support should be matched closely enough that the measured difference can reasonably be attributed to the evaluated system.

The 2024 RAND red-team study found no statistically significant increase in the viability of biological attack plans for teams using the tested language models compared with an internet-only condition (Mouton et al., 2024). The result remains a useful design precedent for controlled baselines, but it is not a current capability ceiling.

Hong et al. subsequently conducted a preregistered, investigator-blinded randomized trial with 153 novice participants using mid-2025 models (Hong et al., 2026, preprint). The primary endpoint showed no significant difference in full workflow completion, while intermediate-step and Bayesian analyses were compatible with a possible modest benefit (Hong et al., 2026, preprint). The study demonstrates why physical-world validation and uncertainty reporting are needed, but it does not exclude uplift for experts, other tasks, or later models.

Statistical Validity

Evaluation plans should define the estimand, effect size of concern, power, stopping rule, multiplicity strategy, missing-data treatment, and uncertainty interval before results are reviewed. Repeated trials on the same item or participant are not independent observations. Multiple metrics should not be interpreted as separate confirmatory tests without accounting for their correlation.

NIST AI 800-2 is an Initial Public Draft addressing the validity, transparency, and reproducibility of automated benchmark evaluations. NIST AI 800-3 is a published statistical-methods report that distinguishes performance on a fixed benchmark from performance generalized to a broader task population. Neither document is a biosecurity-specific operational standard, but both support explicit measurement targets, assumptions, and uncertainty (NIST CAISI, 2026).

Evaluating Safeguard Effectiveness

Safeguards should be evaluated as a stack. Evidence that one layer performs well does not validate the others.

  1. Coverage and precision: Test hazardous requests that should be restricted and legitimate scientific requests that should remain available. Measure both missed hazards and over-refusal. Review the policy taxonomy for omissions as biotechnology and threat models change.
  2. Adversarial robustness: Test skilled human and automated attacks under predeclared time, query, cost, and adaptation budgets. Include multi-turn, multimodal, and tool-enabled conditions where relevant.
  3. Legitimate and exempt access: Examine who can receive reduced restrictions, how identity and purpose are verified, whether permissions are scoped and time-limited, and what compensating controls apply.
  4. Monitoring and remediation: Test detection across interactions, escalation, account action, patch development, regression testing, and time to verified containment.
  5. Independent review: Document reviewer selection, expertise, conflicts, compensation, access to unredacted evidence, analytic autonomy, and publication or redaction rights.

Anthropic’s August 2026 Fable 5 update illustrates the coverage-precision tradeoff in staged deployment. Anthropic reported that, after initially routing almost all biology queries to a less biologically capable model, revising the classifier constitution and training data reduced biology-related fallbacks by approximately 85% across its product surfaces while continuing to route dual-use professional biology and drug-development requests to Opus 5 (Anthropic, August 2026). The announcement did not report paired false-negative, jailbreak, or misuse outcomes. Lower fallback frequency does not establish that safety performance was preserved.

A June 2026 GovAI technical report similarly argues for cross-company comparability, configuration-specific evaluation, dynamic reassessment, and preservation of legitimate scientific access. The report is not peer reviewed and should be treated as a proposed standard rather than an established one (Sudarshan and Righetti, 2026, technical report).

Interpreting Jailbreak Results

A jailbreak is evidence that a tested control can be bypassed under specified conditions. It is not sufficient to count any non-refusal as a successful harmful outcome. The response should be assessed for relevance, correctness, specificity, completeness, usability, and incremental value above the baseline.

StrongREJECT showed that common automated evaluators overstated jailbreak effectiveness compared with human judgments because some attacks bypassed refusal while degrading the model’s ability to provide useful answers (Souly et al., 2024). Biosecurity evaluations should therefore separate:

  • policy bypass: the system did not follow the intended restriction;
  • substantive assistance: the output provided accurate and useful restricted assistance;
  • operational consequence: the assistance reduced a material pathway bottleneck;
  • system failure: other controls did not detect, interrupt, or remediate the event.

The public report should describe these scoring dimensions without publishing reusable attack strings or sensitive outputs.

SecureBio External Review

SecureBio’s July 2026 external review provides a useful case study in separating safeguard claims. It measured Opus 4.6 on 398 BioTIER-refuse prompts and reported 94.2% refusal across ten epochs without jailbreaks (SecureBio, July 2026). Half of the missed prompts came from one topic absent from the classifier’s training constitution (SecureBio, July 2026). This result measured standard-use topic coverage, not adversarial robustness, unguarded-model safety, or legitimate-use utility because paired permit results were not reported in that review.

The review also documented extensive non-public evaluator access. SecureBio reported that Anthropic retained veto and redaction authority but exercised neither. SecureBio therefore marked one AEF-1 transparency requirement as unmet while finding that it retained editorial control over the final report (SecureBio, July 2026). The appropriate conclusion is not that the review lacked value, but that evaluator independence and transparency are multidimensional and should be reported rather than assumed.

The UK AI Security Institute subsequently bypassed safeguards that had resisted thousands of hours of prior human red-teaming by using a new automated black-box method. The finding reinforces that negative red-team results are time-bounded and budget-dependent (UK AISI, February 2026).

Independent Evaluation

External evaluation can reduce correlated blind spots, but “third party” does not by itself establish independence. A credible arrangement should address:

  • evaluator selection and removal;
  • funding and other conflicts;
  • access to the deployed system and relevant internal evidence;
  • ability to test safeguards-disabled or alternative configurations under secure conditions;
  • time, compute, query, and staffing sufficiency;
  • authority to select methods and interpret findings;
  • rights to record disagreement, disclose redactions, and publish bounded conclusions;
  • procedures for handling sensitive findings and verifying remediation.

The UK AI Security Institute’s safeguard principles organize assessment into five stages: define requirements, document the safeguard plan, gather evidence, establish reassessment, and judge whether the evidence and maintenance plan are sufficient (UK AISI, 2025). Independent evaluation strengthens each stage but does not certify that a system is safe.

Lifecycle Monitoring and Remediation

Pre-deployment evaluation is a snapshot. Assurance should continue after deployment through:

  • monitoring for concerning patterns across interactions and accounts;
  • threat intelligence and structured vulnerability intake;
  • incident classification and escalation;
  • preservation of evidence for investigation;
  • temporary restrictions when risk cannot be bounded;
  • patch development and independent regression testing;
  • off-cycle reassessment after capability, safeguard, access, or threat-model changes;
  • documented closure only after the failure is no longer reproducible under the relevant conditions.

When a model or agent can affect connected laboratory systems, the same lifecycle should feed the laboratory’s incident and near-miss process. The canonical control model is Cyberbiosecurity for Connected Laboratories.

Anthropic’s Responsible Scaling Policy was revised repeatedly during 2026 and reached version 3.4 in July. The current policy adds off-cycle risk reporting, visible indications of redaction, and external review provisions for unredacted portions of risk reports. These are voluntary company commitments and should be described as governance practice, not binding regulation (Anthropic, July 2026).

For providers of general-purpose AI models with systemic risk, Article 55 of the EU AI Act requires model evaluation using standardized protocols and current tools, documented adversarial testing, systemic-risk assessment and mitigation, serious-incident reporting, corrective measures, and cybersecurity protections. These are lifecycle obligations, not a biosecurity-specific test protocol (Regulation (EU) 2024/1689, Article 55).

Decision and Reporting Framework

A red-team report should support a decision rather than end with a list of attacks attempted. The decision record should contain:

Element Required content
Scope Exact model, system configuration, access tier, tools, safeguards, and evaluation date
Threat model Actor, baseline resources, pathway, bottleneck, severity threshold, and adversary budget
Methods Tasks, elicitation, controls, attempts, scoring, graders, contamination checks, and statistical plan
Findings Capability, uplift, safeguard, and operational findings reported separately
Uncertainty Confidence intervals, lower-bound limitations, untested conditions, and evaluator disagreements
Safeguard assessment Coverage, robustness, legitimate access, monitoring and remediation, and independence
Residual risk Remaining pathways, control dependencies, and consequence if controls fail
Decision Deploy, restrict, add safeguards, defer, or stop, tied to predeclared criteria
Follow-up Owners, remediation evidence, retest conditions, monitoring metrics, and reassessment triggers

An assurance claim should not be carried forward to a materially different model, policy, tool configuration, access mode, or monitoring stack without new evidence. Every conclusion should identify the system that was actually tested.

Controlled Evaluation Checklist

Before testing:

  • Confirm authorization, institutional oversight, information-hazard controls, and stop conditions.
  • Freeze the threat model, model version, deployment configuration, baselines, and decision thresholds.
  • Define evaluator access, budgets, expertise, conflicts, and secure evidence handling.
  • Preregister confirmatory analyses where human or agent uplift is being estimated.

During testing:

  • Separate latent capability from deployed safeguard behavior.
  • Test benign scientific utility alongside hazardous-request coverage.
  • Include repeated, adaptive, multi-turn, account-level, and tool-enabled conditions when relevant.
  • Record all attempts, configuration changes, exclusions, incidents, and deviations.
  • Stop or contain testing if evidence handling or execution conditions exceed the approved boundary.

After testing:

  • Grade substantive assistance rather than refusal bypass alone.
  • Report capability, uplift, safeguard, operational-risk, and governance findings separately.
  • State uncertainty, lower-bound conditions, and evidence gaps.
  • Assign remediation owners and require regression testing.
  • Record a residual-risk decision and the conditions that trigger reassessment.

Limitations of Red-Teaming

Red-teaming cannot exhaust an open-ended attack surface. Results vary with evaluator skill, access, incentives, test budget, and knowledge of the system. Private tests improve resistance to benchmark contamination but limit public reproducibility. Public tests improve scrutiny but can become training data or expose defensive gaps. Physical validation can improve realism while also increasing safety, legal, and information-hazard burdens.

These limitations do not make red-teaming optional. They define the claims that red-teaming can support. The strongest defensible conclusion is bounded: under a specified threat model, system configuration, evaluator team, and resource budget, the evaluation found specified capabilities or failures, measured specified safeguards, and left stated residual uncertainties.

For the broader state of biological AI capability evidence, see AI as a Biosecurity Risk Amplifier. For information-hazard mechanisms and model-level safeguards, see LLMs and Information Hazards. For agent-specific attack surfaces, see Autonomous AI Agents. For governance requirements, see Policy Frameworks for AI-Bio Convergence.