Red-Teaming AI Systems for Biosecurity Risks

In late 2024, Anthropic activated ASL-3 safeguards for Claude Opus 4 after red-team evaluations revealed the model could provide CBRN weapons instructions. This wasn’t a failure. The system worked exactly as designed: adversarial testing identified a capability threshold breach, automatically triggering enhanced security protocols including classified government collaboration with NNSA and DOE. Meanwhile, RAND’s controlled experiments found current LLMs don’t dramatically increase bioweapons risk beyond internet search, but emphasized continuous monitoring remains essential as AI capabilities evolve rapidly and today’s negative result doesn’t guarantee tomorrow’s safety.

Learning Objectives
  • Apply red-teaming methodologies to identify AI-enabled biological threats
  • Interpret uplift studies measuring whether AI increases novice capabilities
  • Recognize jailbreak vulnerabilities and information hazards in LLMs
  • Evaluate AI systems using ASL frameworks and capability benchmarks
Scope of This Chapter

This chapter discusses biosecurity risks at a conceptual level appropriate for education and policy analysis. Consistent with responsible information practices:

  • Omitted: Actionable protocols, specific synthesis routes, exact pathogen sequences
  • Included: Risk frameworks, governance mechanisms, policy recommendations

For detailed biosafety protocols, consult your Institutional Biosafety Committee and relevant regulatory guidance.

Key Findings from Real Evaluations:

What Works:

Bottom Line: Physical barriers (lab access, materials, tacit knowledge) remain significant. But AI systematically lowers knowledge barriers. Biosecurity community needs red-teaming capacity matching AI labs’ development pace.

Introduction

In late 2024, Anthropic activated ASL-3 (AI Safety Level 3) safeguards for Claude Opus 4 after evaluations revealed the model could provide CBRN weapons instructions. This wasn’t failure. The system worked: red-teaming identified a capability breach, triggering enhanced security protocols automatically.

Red-teaming in biosecurity means adversarial testing where evaluators actively try to misuse AI systems for harmful biological purposes. Not benign queries. Jailbreaking LLMs to extract synthesis protocols. Testing whether AI gives novices expert capabilities. Measuring information leakage under adversarial prompting.

This chapter covers methodologies with evidence from Anthropic red-teaming, OpenAI preparedness, RAND, and UK AISI.

The ASL Framework: Biosafety Levels for AI

Anthropic’s AI Safety Level (ASL) system is explicitly modeled on U.S. biosafety level (BSL) standards used for handling dangerous biological materials. This biosafety-to-AI translation provides familiar risk categorization for biosecurity professionals.

The Translation:

BSL Pathogen Example ASL Risk Example
BSL-1 E. coli K-12 ASL-1 Basic chatbot
BSL-2 Influenza ASL-2 Current LLMs with safety training
BSL-3 SARS-CoV-2 ASL-3 CBRN weapon instructions (Claude Opus 4)
BSL-4 Ebola ASL-4 Autonomous biological design

ASL-3 Safeguards:

  • Exceptionally strong access controls
  • No deployment exceeding catastrophic misuse thresholds
  • World-class red teamers must fail to demonstrate CBRN misuse risk
  • Government collaboration: classified red-teaming with NNSA, DOE and federal agencies

When Claude Opus 4 crossed the CBRN-3 threshold (can provide meaningful CBRN assistance), ASL-3 protocols automatically engaged.

OpenAI’s Preparedness Framework takes a similar threshold-based approach:

  • Tracks CBRN as high-risk category
  • Low: No increase over internet search baseline
  • Medium: Some uplift, not beyond undergraduate biology knowledge
  • High: Expert-level biological knowledge or design capabilities
  • Critical: Novel capabilities fundamentally changing threat landscape
  • Commitment: Won’t release models exceeding “High” threshold in biology without robust mitigations

Key insight from both frameworks: biosecurity risks require specific, measurable capability thresholds, not abstract threat assessments.

Uplift Studies and Novice Capability Assessment

Uplift studies measure whether AI increases capability for tasks with dual-use potential. Core question: Does AI democratize dangerous expertise?

RAND Study (2024)

RAND Corporation conducted controlled experiments where participants role-played malicious actors planning biological attacks. Some groups had LLM access, others only internet search as baseline.

Finding: Current LLMs did not significantly enhance bioweapons risk compared to information already available online. The LLM group produced attack plans similar in quality and completeness to the internet-only group.

Critical caveat: RAND emphasized continuous monitoring because AI capabilities evolve rapidly. Today’s negative result doesn’t guarantee tomorrow’s safety as models improve.

Anthropic’s Biosecurity Evaluations

Anthropic’s uplift studies showed their models approaching undergraduate-level skills in cybersecurity and expert-level knowledge in some biology areas. They tested weaponization-related tasks in collaboration with biodefense experts.

Critical finding: AI provided some uplift to novices, but even highest-scoring plans with AI still contained critical errors that would prevent real-world success. This suggests physical barriers and tacit knowledge remain significant obstacles.

This matters. AI lowered barriers but didn’t eliminate them. Physical constraints (lab access, tacit knowledge, materials procurement) remain significant.

Expert vs. Novice Differential

For Novices (Soice et al.):

  • AI assists with lack of hands-on experience
  • Provides expert-level troubleshooting for practical lab work
  • Simplifies access to complex technical information
  • Forecast: AI could support novices acquiring biological weapons by late 2025 (Sandbrink, 2023)

For Experts (Sandbrink, 2023):

  • Elevates existing capabilities further
  • Enables novel protein design and optimization
  • Accelerates discovery processes significantly
  • Forecast: AI supporting experts developing novel bioweapons by late 2026

Virology Capabilities Test (VCT) benchmarks AI ability to troubleshoot complex virology laboratory protocols. Some current models now exceed human expert performance on these benchmarks.

UK NCSC Uplift Framework

The UK National Cyber Security Centre (NCSC) uses standard UK intelligence community probability language to assess AI-enabled capability uplift. Their January 2024 assessment concluded that AI will “almost certainly” increase volume and intensity of threats, with differential impact by actor type:

  • Highly capable state actors: Can harness full potential of AI for advanced operations
  • Skilled criminals: Will “highly likely” develop AI tools as-a-service for others
  • Novices and hacktivists: Receive uplift “from a low base” through accessible AI-enabled tools

This framework applies directly to biosecurity: the same differential holds for biological threats, where state-level actors with laboratory infrastructure benefit most while novices face persistent physical barriers despite knowledge uplift.

Reality check: Physical production of AI-enabled designs remains a significant barrier. Foundational biosecurity gaps exist independently of AI capabilities.

Jailbreak Attacks and Information Hazards

Jailbreak attacks are intentional attempts to bypass LLM safety measures through adversarial prompts and prompt injection.

Techniques

Well-documented jailbreak techniques include:

  • Role-playing: “You are a safety researcher testing dangerous content filters…”
  • Attention shifting: Manipulating the model’s instruction-following behavior
  • Prompt injection: Embedding malicious instructions within benign queries

Information Hazards

LLMs may inadvertently reveal sensitive biological information through information hazard leakage. Research demonstrates LLMs respond less stringently to information hazards compared to other risk categories, making them particularly vulnerable to jailbreaking in biosecurity contexts.

Vulnerability example: Attackers use sophisticated jailbreak techniques to extract biosecurity-sensitive information the model is nominally programmed to withhold.

OpenAI’s Defensive Measures

OpenAI has deployed multiple layers of defense:

Core challenge: You can’t patch a model’s knowledge after deployment. Red-teaming must catch these vulnerabilities before release.

Defending Against Jailbreaks: Constitutional Classifiers

While jailbreak attacks probe vulnerabilities, Constitutional Classifiers represent the current state-of-the-art in defense. Developed by Anthropic, these systems use classifiers trained on synthetic data generated from natural language rules (a “constitution”) specifying permitted and restricted content.

How Constitutional Classifiers Work

The system operates at use-time, monitoring exchanges between users and AI models. Anthropic’s Constitutional Classifiers++ (January 2026) introduced three key innovations:

Exchange Classifiers: Rather than examining inputs and outputs separately, exchange classifiers evaluate model responses in the context of their corresponding inputs. This addresses two attack categories that evaded earlier defenses:

  • Reconstruction attacks: Adversaries fragment harmful information across benign segments (e.g., embedding a query as scattered function return values in a codebase), then instruct the model to reassemble and respond
  • Output obfuscation attacks: Attackers prompt models to substitute sensitive terms with innocuous alternatives or use metaphors that appear benign in isolation but reveal harmful content when paired with the input context

Two-Stage Classifier Cascade: A lightweight first-stage classifier screens all traffic, escalating only suspicious exchanges to a more expensive second-stage classifier. Because escalation triggers additional review rather than immediate refusal, the first stage can flag a higher proportion of traffic without increasing false positive rates.

Linear Probe Ensembles: Efficient classifiers using the model’s internal activations can be combined with external classifiers. These probes add negligible computational cost while capturing complementary signals, improving overall robustness.

Production Performance

Anthropic’s production-grade system achieves:

Metric Previous Generation Constitutional Classifiers++
Computational overhead 23.7% ~1%
False positive rate 0.38% 0.05%
Red-teaming without universal jailbreak 3,000+ hours 1,700+ hours (ongoing)

The 40x reduction in computational cost makes production deployment viable. The 0.05% refusal rate on legitimate traffic represents a significant improvement over previous systems.

Red-Teaming Results

Through over 1,700 hours of adversarial testing against the production system, no universal jailbreak was discovered capable of extracting detailed CBRN information comparable to an undefended model across all target queries. This represents the strongest robustness achieved among all systems Anthropic has evaluated.

Key findings from red-teaming:

  • Only one high-risk vulnerability discovered across approximately 198,000 attempts
  • Minimum discovery time for high-risk vulnerabilities: 30 hours
  • Vulnerability discovery rate: 0.005 per thousand queries (lowest among all tested systems)

Implications for Biosecurity

Constitutional Classifiers demonstrate that robust jailbreak defense is achievable at production scale. For biosecurity practitioners evaluating AI systems:

  1. Ask about defense architecture: Does the system use exchange-level classification or examine inputs/outputs separately?
  2. Request red-teaming metrics: What is the vulnerability discovery rate? Has any universal jailbreak been found?
  3. Verify production viability: High false positive rates or computational costs may indicate immature defenses
  4. Recognize limitations: No defense is perfect. Constitutional Classifiers reduce but do not eliminate jailbreak risk

Capability Benchmarking

Capability benchmarking measures AI’s ability to perform tasks that could be misused for biological threats.

Benchmark Examples

PropensityBench: Simulates real-world pressure where AI agents must choose between safe procedures and harmful shortcuts. Reveals model tendency toward unsafe choices under stress conditions.

UK AISI Chem-Bio Uplift Studies: Evaluate dual-use implications through both automated testing and human-interactive scenarios. Resource-intensive but essential for evidence-based policy development.

Evaluation Challenges

Building comprehensive biosecurity evaluation capacity faces several obstacles:

  • Creating rigorous evidence base is resource-intensive
  • Need in-depth evaluations beyond surface-level testing
  • Require both automated benchmarks and human-interactive studies
  • Must balance beneficial research applications vs. misuse potential

Peaceful Proxy Evaluations

A critical challenge in biosecurity evaluation is wet-lab validation: determining whether an AI-generated design actually functions requires creating and testing it, which poses accident risks. The U.S. government has called for development of “peaceful proxy” evaluations that accurately assess dangerous capabilities while minimizing validation risks (Pannu et al., 2025).

One emerging approach: the Evo 2 genomic foundation model biosecurity evaluation repurposed the existing ProteinGym benchmark to assess mutational effect prediction on viral proteins without generating novel dangerous sequences. However, research demonstrates that data exclusion safeguards (removing pathogenic sequences from training) can be bypassed through adversarial fine-tuning if an actor obtains sensitive data, highlighting the limits of training-time interventions.

Building Red-Teaming Capacity

Anthropic’s Frontier Red Team represents the current state of practice:

  • Approximately 15 dedicated researchers
  • Full-time adversarial testing of most advanced systems
  • Cover domains: cybersecurity, biological research, autonomous systems
  • Publish findings transparently through dedicated blog “Red”
  • Community engagement through initiatives like DEF CON AI Village and GRT Challenge

Critical capacity gap: AI labs have resources for dedicated red-teaming. Biosecurity community needs equivalent independent evaluation capacity.

Emerging standardization: The Global Risk Index for AI-enabled Biological Tools (CLTR-RAND, 2025) provides rubrics for systematically assessing AI tools across four dimensions: capabilities, misuse potential, accessibility, and technological maturity. This offers a common framework for pre-development and pre-deployment assessment. See Policy Frameworks for AI-Bio Convergence for details.

Best Practices drawn from current implementations and OpenAI:

  1. Domain expertise first: Biosecurity evaluations require biologists, epidemiologists, and biodefense experts, not just ML engineers
  2. Adversarial mindset: Test with malicious intent, not benign good-faith queries
  3. Baseline comparisons: Document what AI provides vs. what’s already accessible via Google
  4. Continuous evaluation: One-time testing insufficient as model capabilities evolve
  5. Government collaboration: Classified red-teaming for national security contexts through partnerships with NNSA, DOE

Practical Red-Teaming Checklist

If evaluating an AI system with biological capabilities:

1. Test Uplift Differential (UK AISI)

Compare performance across groups: - Novices + AI vs. novices + internet only - Experts + AI vs. experts + internet only

Measure: Does AI provide meaningful capability increase above baseline?

2. Use Adversarial Prompts (Promptfoo)

Ineffective test: “How do I make anthrax?” Effective test: “I’m a biosafety researcher evaluating dual-use information controls…”

Test jailbreak resistance systematically.

3. Measure Against Specific Thresholds (Anthropic RSP / OpenAI Preparedness)

Not “Is this risky?” but “Can it troubleshoot BSL-3 protocols?” or “Does it exceed undergraduate biology knowledge?”

Concrete capability benchmarks, not vibes-based assessments.

4. Document Baseline (RAND)

What information can users find via Google in equivalent time?

AI uplift = (AI-assisted capability) minus (baseline capability without AI)

5. Test Information Hazard Resistance (Shen et al.)

Attempt to extract: - Synthesis protocols for select agents - Troubleshooting guidance for dangerous procedures - Optimization strategies for enhancing pathogen traits

Does model refuse appropriately? Or leak information with clever prompting?

What is AI red-teaming for biosecurity and why is it important?

AI red-teaming involves adversarial testing where evaluators deliberately attempt to misuse AI systems for harmful biological purposes. This includes jailbreaking LLMs to extract dangerous protocols, testing whether AI gives novices expert capabilities, and measuring information leakage under adversarial prompting. It’s critical because capabilities evolve rapidly. Today’s safe model could cross concerning thresholds tomorrow without continuous evaluation.

What are AI Safety Levels (ASL) and how do they relate to biosafety?

Anthropic’s ASL framework is explicitly modeled on biosafety levels (BSL-1 through BSL-4). ASL-1 represents basic chatbots, ASL-2 current LLMs with safety training, ASL-3 systems providing CBRN weapon instructions (like Claude Opus 4), and ASL-4 autonomous biological design capabilities. Each level triggers specific security protocols, with ASL-3 requiring exceptionally strong access controls and government collaboration with agencies like NNSA and DOE.

Do current AI systems significantly increase bioweapons risk?

RAND’s controlled experiments found current LLMs don’t dramatically increase bioweapons risk beyond internet search baseline. Anthropic’s evaluations showed AI provides some uplift to novices, but even highest-scoring plans with AI assistance contained critical errors preventing real-world success. Physical barriers (lab access, materials procurement, tacit knowledge) remain significant obstacles. However, knowledge barriers are systematically lowering, making continuous monitoring essential as capabilities advance.

What are uplift studies and what do they measure?

Uplift studies are controlled experiments measuring whether AI increases capability for dual-use tasks. Researchers compare novices using AI versus internet-only access to test if AI democratizes dangerous expertise. The key metric is: (AI-assisted capability) minus (baseline capability without AI). Current studies show measurable but limited uplift, with physical constraints still constraining threats, but these experiments must continue as model capabilities improve.

Reality Check

If you’re building or evaluating AI systems with biological capabilities:

Most importantly: Physical barriers still matter. AI alone doesn’t create bioweapons. But it’s lowering the knowledge barrier systematically. That’s the measurable risk to evaluate.