Policy Frameworks for AI-Bio Convergence
AI-bio convergence creates governance gaps that existing frameworks were never designed to address. Biosecurity policy assumes physical materials; AI policy addresses digital systems. Neither fully covers what happens when AI enables biological work, creating an ungoverned space where novel threats emerge faster than regulations can adapt.
- Map existing biosecurity governance frameworks (BWC, DURC, select agent regulations) to AI-enabled biological risks
- Evaluate proposed policy interventions including compute governance, DNA synthesis screening, and model access tiers
- Analyze the roles of different stakeholders (governments, companies, researchers, international bodies) in AI-bio governance
- Identify governance gaps where AI and biosecurity converge
- Apply a layered defense framework to policy design for emerging biological risks
Introduction: Governing the Ungoverned Space
Throughout this handbook, we have traced how AI intersects with biological risks: providing information (LLMs and Information Hazards), enabling design (AI-Enabled Pathogen Design), and potentially executing experiments (Cloud Labs and Automated Biology). Each chapter has noted governance gaps.
This chapter addresses those gaps directly.
The challenge is that AI-bio convergence creates risks that straddle traditionally separate policy domains:
- Biosecurity policy assumes physical materials (pathogens, equipment) and trained personnel
- AI policy assumes digital systems, computational resources, and model outputs
- Neither fully addresses what happens when AI systems enable biological work
We are currently governing this intersection through a patchwork of existing frameworks, voluntary commitments, and emerging proposals. Some of this is working. Much of it is not.
We are in a relatively narrow window where proactive governance is possible.
- AI capabilities for biological design are advancing but not yet mature
- Cloud lab automation is growing but not yet ubiquitous
- The biosecurity community is increasingly engaged with AI risks
- Major AI companies are receptive to governance discussions
Once a serious incident occurs, policy will shift to reactive mode. History suggests reactive biosecurity policy is often poorly calibrated - either too restrictive (hindering legitimate research) or too narrow (missing the actual threat). The goal is to get governance right proactively.
Existing Frameworks and Their Limitations
The Biological Weapons Convention (BWC)
The Biological Weapons Convention, which entered into force in 1975, prohibits the development, production, and stockpiling of biological weapons. It represents the foundational international norm against bioweapons.
What it covers: - General prohibition on bioweapon development - Commitment to peaceful use of biology - Framework for international cooperation
What it misses: - No verification mechanism (unlike the Chemical Weapons Convention) - No provisions addressing AI, computational biology, or information hazards - Relies on national implementation, which varies widely - Does not address private sector or non-state actors effectively
The BWC’s strength is its normative power - it establishes that bioweapons development is unacceptable. Its weakness is enforcement. For AI-bio convergence, the greater weakness is conceptual: the BWC was written for a world of state programs, physical pathogens, and traditional laboratories.
RAND analysts have noted the Pandemic Accord’s dangerous blind spot: the WHO resolution barely acknowledges laboratory biosafety and biosecurity as threats to global health security. They recommend mandatory reporting for laboratory accidents and Joint External Evaluation-style assessments of laboratory capabilities, noting that three-quarters of BSL4 laboratories are in urban areas where accidents could pose community risks.
Select Agent Regulations
In the United States, the Federal Select Agent Program regulates possession and use of dangerous pathogens and toxins. Similar programs exist in other countries.
What it covers: - Registration of entities possessing select agents - Personnel reliability screening - Physical security requirements - Transfer controls
What it misses: - Information about how to create or modify pathogens (not regulated) - AI-generated designs that could produce equivalent dangers - Synthetic constructs that do not match listed agents but could be equally harmful
Select agent regulations remain essential, but they address physical possession rather than capability. A world where AI can design novel pathogens with equivalent danger to select agents is not one these regulations were designed for.
RAND’s 2025 assessment of the Federal Select Agent Program identifies gaps across four categories: identification of new agents of concern, speed of response to novel pathogens, transparency of designation criteria, and institutional culture prioritizing security over safety. Emerging biotechnologies, including AI, call for proactive FSAP review.
Dual-Use Research of Concern (DURC) Policies
NIH policies on DURC require institutional review of federally funded research that could be misused to threaten public health or national security.
What it covers: - Institutional biosafety committee review - Risk-benefit assessment - Risk mitigation measures
What it misses: - Research not federally funded - AI system outputs (not “research” in the traditional sense) - Work conducted outside traditional institutions (startups, cloud labs) - International actors not subject to NIH oversight
DURC policies represent a mature approach to dual-use governance, but their applicability to AI-generated biological designs is unclear. Is an AI output “research”? Who is the “researcher” when an AI designs a sequence?
In May 2024, the US government announced an updated Policy for Oversight of DURC and Pathogens with Enhanced Pandemic Potential (PEPP), effective May 2025. This policy:
- Expands and unifies previous DURC and P3CO policies
- Applies to all federal departments and agencies funding life sciences research
- Can require institutions to implement oversight of non-federally funded research as a condition of receiving any federal funding
However, the policy remains focused on physical experiments with pathogens, not AI systems that provide advice or design capabilities.
The 2024 OSTP Framework for DNA Synthesis Screening
The 2024 OSTP Framework for Nucleic Acid Synthesis Screening represents the most significant recent biosecurity policy development in the United States.
What it covers: - Expectations for federally funded synthesis providers - Baseline screening standards - Customer verification requirements - Guidance on best practices
What it misses: - Voluntary framework (no enforcement mechanism) - International providers not covered - Novel AI-designed sequences that evade pattern-matching - Function-based screening not required
The OSTP Framework is a meaningful step forward, but it remains voluntary and focused on traditional sequence-matching. As AI-Enabled Pathogen Design noted, AI can design proteins with conserved function but divergent sequence - potentially evading current screening.
Executive Order 14292 (May 2025) directed OSTP to revise the 2024 Framework within 90 days, requiring enforcement mechanisms rather than voluntary attestations. The July 2025 AI Action Plan further specified that federal funding recipients must use synthesis providers with robust screening and customer verification, verified through compliance mechanisms.
The AI Action Plan also calls for a data-sharing mechanism between synthesis providers to detect “split ordering,” where malicious actors distribute dangerous sequences across multiple companies to evade detection. This addresses a known gap in current screening: orders split below detection thresholds at each provider but assembling into dangerous constructs.
However, the 90-day revision deadline passed without a published updated framework. Implementation timelines remain uncertain.
Proposed Policy Interventions
The biosecurity and AI policy communities have proposed numerous interventions. Here we assess the most prominent.
Compute Governance
The concept: Large AI training runs require substantial computational resources. Monitoring and potentially regulating access to large-scale compute could provide visibility into who is training powerful models.
Proposed mechanisms: - Reporting requirements for compute usage above thresholds - Know-your-customer standards for cloud compute providers - Export controls on advanced AI chips
Assessment: - Strengths: Addresses model development rather than just deployment; difficult to evade at scale - Weaknesses: Biological design may not require frontier-scale compute; dual-use with legitimate applications; could push development to less transparent jurisdictions
Current implementations:
| Jurisdiction | Threshold | Requirements | Status |
|---|---|---|---|
| US (EO 14110) | 10^26 FLOPs | Reporting, safety testing for CBRN | Rescinded January 2025; replaced by EO 14179 |
| EU AI Act | 10^25 FLOPs | Systemic risk assessment, model evaluation | In force August 2024 |
| California SB 53 | 10^26 FLOPs | Safety framework, incident reporting | Enacted 2025; subject to federal preemption challenge per December 2025 EO |
The release of OpenAI’s o1 model in late 2024 highlighted a significant limitation of compute thresholds. Unlike previous frontier models, o1’s capabilities derive substantially from “inference compute” - computational resources used when the model responds to queries - rather than training compute alone.
This means models trained with relatively modest compute can achieve frontier-level performance on some tasks. Compute governance remains useful as a first filter, but must be combined with capability evaluations.
The 2025 NASEM report noted that current biological design tools do not require frontier-scale compute. This limits the applicability of compute governance specifically for AI-bio risks, though it may be valuable for broader AI safety.
Enhanced DNA Synthesis Screening
The concept: Extend and strengthen DNA synthesis screening to address AI-designed sequences that evade traditional pattern-matching.
Proposed mechanisms: - Function-based screening (not just sequence matching) - Universal coverage including international providers - Mandatory rather than voluntary standards - Coverage of benchtop synthesis equipment
Assessment: - Strengths: Targets a genuine chokepoint; builds on existing industry practice - Weaknesses: International coordination required; benchtop devices harder to control; ongoing arms race between design and screening
SecureDNA and similar initiatives represent promising approaches to function-based screening. The International Biosecurity and Biosafety Initiative for Science (IBBIS) developed the Common Mechanism - a free, open-source, globally available tool for DNA sequence screening. The policy challenge is achieving universal adoption across providers globally.
The “Open Weights” Debate
The central tension: When a company trains a powerful AI model, they have two fundamental choices:
- Closed API: The model stays on their servers. Users send queries, receive answers, and can be monitored or blocked for concerning requests.
- Open Weights: The model’s “brain” (weights) is published online. Anyone can download it and run it locally.
The biosecurity concern: Once a model is open weights, safety guardrails can be stripped out. A “jailbroken” open model can be retrained to remove all refusals.
The counter-argument: Open science drives innovation. Restricting access means only large tech companies and governments can use the best tools.
Model Access Tiers
The concept: Implement differential access to AI models based on capability level and user verification.
Proposed mechanisms: - Open access for lower-capability models - API-only access for more capable models (with monitoring) - Restricted access for frontier capabilities
Assessment: - Strengths: Allows broad access while controlling high-risk capabilities; aligns with industry practice - Weaknesses: Open-source models exist outside this framework; capability thresholds difficult to define; may not prevent advanced actors
Major AI companies have already implemented tiered access. The policy question is whether voluntary measures are sufficient, and how to address the open-source ecosystem.
Pre-Training Safety Evaluation
The concept: Require safety evaluation of AI models before training or deployment, specifically including biosecurity assessments.
Proposed mechanisms: - Red-teaming requirements for biological capabilities - Structured access for external evaluators - Defined capability thresholds that trigger additional scrutiny
Assessment: - Strengths: Catches issues before deployment; creates accountability - Weaknesses: Evaluation methodology still maturing; may not catch all risks; adversarial actors can evade
Red-Teaming AI Systems for Biosecurity Risks addresses methodology. The policy question is whether to mandate such evaluations and how to structure oversight.
Global Risk Index for AI-Enabled Biological Tools
The concept: A structured framework for systematically assessing AI-enabled biological tools based on capabilities, misuse potential, accessibility, and technological maturity.
The Global Risk Index for AI-enabled Biological Tools (CLTR-RAND, September 2025) provides the first comprehensive rubric for evaluating these tools before development, deployment, and release. This addresses a gap where individual companies assessed risks using incompatible internal frameworks.
The framework assesses four dimensions:
- Capability: What can the tool do? Does it design sequences, predict structures, troubleshoot protocols?
- Misuse potential: How readily could capabilities be applied to harmful purposes?
- Accessibility: Is it open-source, API-access, or restricted? What verification is required?
- Technological maturity: Prototype, production-ready, or widely deployed?
Key recommendations from the Index:
- Pre-development assessment: Developers and funders should use the rubrics to assess tools for misuse-relevant capabilities before funding and development, not just before release
- Managed access via KYC: Implement Know Your Customer principles for tools with significant misuse-relevant capabilities, differentially prioritizing defensive applications like medical countermeasures
- Safeguards by design: Funders should enable developers to embed technical safeguards into tools from the start, with non-safeguarded versions available only to legitimate defensive researchers under secure conditions
- Responsible innovation culture: Regular convenings between developers, funders, and government experts to coordinate assessments and share best practices
- Ongoing monitoring: Government and independent experts should refresh the Index every six months to avoid strategic surprises
Assessment:
- Strengths: Provides standardized rubric across the field; addresses pre-development stage often missed by deployment-focused policies; emphasizes defense prioritization
- Weaknesses: Voluntary adoption; international coordination required; rubric scoring may be subjective without calibration
The Global Risk Index complements existing frameworks (ASL, Preparedness Framework) by focusing specifically on biological AI tools rather than general-purpose AI, and by targeting the pre-development phase where decisions about whether to build have the greatest impact.
Responsible Scaling Policies
The concept: AI companies commit to pausing or restricting deployment if models exceed defined capability thresholds.
Examples: - Anthropic’s Responsible Scaling Policy - OpenAI’s Preparedness Framework
Assessment: - Strengths: Industry leadership; creates internal governance structures - Weaknesses: Voluntary; may not bind all actors; thresholds not standardized
When assessing AI-bio governance proposals, ask:
1. Does it address the right intervention point? - Is this targeting capability development, deployment, or use? - Does it catch the actors of concern?
2. Is it enforceable? - What is the compliance mechanism? - Who monitors and enforces?
3. Does it scale internationally? - Can it be adopted globally? - Does it create regulatory arbitrage opportunities?
4. Does it preserve legitimate use? - What is the burden on researchers and companies? - Does it hinder beneficial applications?
5. Is it adaptive? - Can it evolve as technology changes? - Are there sunset provisions or review mechanisms?
Stakeholder Roles and Responsibilities
Effective governance requires coordinated action across multiple stakeholders.
National Governments
Key roles: - Setting regulatory frameworks - Funding biosecurity research - International negotiations - Enforcement and oversight
Current gaps: - No clear lead agency for AI-bio convergence in most countries - Biosecurity and AI policy often housed in different agencies - Limited technical capacity for evaluation
AI Companies
Key roles: - Implementing safety measures in model development - Conducting pre-deployment biosecurity evaluation - Responding to identified risks - Sharing threat information
Current gaps: - Voluntary commitments vary in rigor - Open-source ecosystem operates outside company governance - Competitive pressures may limit caution
Research Community
Key roles: - Developing evaluation methodologies - Identifying emerging risks - Publishing responsibly on dual-use topics - Informing policy with technical expertise
Current gaps: - Academic incentives may not align with security - Publication norms still evolving for AI-bio - Limited funding for biosecurity-focused AI safety research
International Bodies
Key roles: - Coordinating standards across jurisdictions - Facilitating information sharing - Providing technical assistance - Monitoring compliance
Emerging mechanisms:
- International Network of AI Safety Institutes: Launched November 2024, this network coordinates evaluation methodologies and shares information across national AI institutes. Note: Both the UK AI Safety Institute (renamed to AI Security Institute in February 2025) and US AI Safety Institute (renamed to Center for AI Standards and Innovation in June 2025) have been rebranded, though the international network continues.
- G7 Hiroshima AI Process: Launched February 2025, this voluntary mechanism invites organizations developing advanced AI to complete transparency questionnaires published on an OECD platform.
- NTI AIxBio Global Forum: Convenes experts to develop recommendations for safe AI-life sciences convergence.
- WHO Guidance on LMMs for Health (2024): Establishes six ethical principles for AI governance and a value chain framework assigning responsibilities to developers, providers, and deployers.
The WHO guidance on large multi-modal models (2024) establishes six consensus ethical principles applicable to AI governance:
- Protect autonomy: Humans remain in control of health-care systems and medical decisions
- Promote human well-being, human safety, and the public interest
- Ensure transparency, explainability, and intelligibility
- Foster responsibility and accountability
- Ensure inclusiveness and equity
- Promote AI that is responsive and sustainable
The guidance frames governance across the AI value chain: developers who build foundation models, providers who integrate them into applications, and deployers who use them in practice. Each stage has distinct responsibilities and risks that governance must address separately.
While the WHO guidance focuses on healthcare applications broadly, these principles apply directly to AI-bio convergence governance, where accountability across the value chain is similarly fragmented.
Current gaps: - BWC has no institutional capacity for AI issues - No international body owns the AI-bio intersection - Limited enforcement mechanisms - Participation remains limited primarily to advanced industrial economies
AI-bio governance suffers from a classic coordination problem:
- No single stakeholder can solve it alone
- Each stakeholder faces different incentives
- First-movers may bear costs while free-riders benefit
- International coordination is particularly difficult
This does not mean coordination is impossible - DNA synthesis screening emerged through industry consortium action. But it does mean governance will require sustained effort across multiple actors.
The geopolitical reality: AI is viewed as a strategic military asset. Nations are hesitant to slow down their own development for “safety” if they think rivals are racing ahead. However, no nation wants a biological accident - a pandemic released in one country will spread to its rivals. This shared vulnerability is the only leverage we have for global cooperation.
A Layered Defense Framework
Given the limitations of any single intervention, effective governance requires layered defense - multiple overlapping controls that together reduce risk even when individual controls fail.
Layer 1: Capability Restrictions
Goal: Limit who can develop or access dangerous capabilities
Mechanisms: - Compute governance for training - Model access tiers for deployment - Responsible scaling commitments
Layer 2: Use-Time Controls
Goal: Detect and prevent misuse during model use
Mechanisms: - Content filtering for harmful requests - Monitoring for concerning patterns - Rate limiting and anomaly detection
Layer 3: Materials Chokepoints
Goal: Prevent translation of AI outputs into physical threats
Mechanisms: - DNA synthesis screening - Cloud lab verification and protocol review - Equipment access controls
Layer 4: Detection and Response
Goal: Detect misuse when prevention fails and respond effectively
Mechanisms: - Enhanced biosurveillance - Attribution capabilities - Countermeasure development - Incident response frameworks
Each layer has weaknesses:
- Capability restrictions can be evaded by determined actors
- Use-time controls can often be bypassed
- Materials chokepoints have coverage gaps
- Detection and response come after harm begins
But together, these layers create multiple barriers. An actor would need to evade all four layers to translate AI capabilities into catastrophic harm. This defense-in-depth approach is more robust than relying on any single control.
Liability for AI-Enabled Biological Harm
When AI systems contribute to biological harm, who bears responsibility? The WHO guidance on LMMs (2024) identifies liability as a critical governance gap: “errors, misuse and ultimately harm to individuals are inevitable,” yet existing liability frameworks were not designed for AI systems with multiple actors across a value chain.
The Liability Problem in AI-Bio Convergence
Consider three scenarios:
An LLM provides synthesis instructions that an attacker uses to produce a biological agent. Is the model developer liable? The company that deployed it? The cloud provider hosting it?
DNA synthesis screening fails because an AI-designed sequence evaded pattern-matching. Who is responsible: the AI tool that generated the evasive design, the screening provider, or the synthesis company?
A cloud lab executes a dangerous protocol submitted through an AI agent. Liability could fall on the AI developer, the protocol’s human requester, or the cloud lab operator.
In each case, responsibility is distributed across the AI value chain, making attribution difficult and creating gaps where no party is clearly accountable.
Potential Liability Frameworks
WHO proposes three approaches that could apply to AI-bio governance:
1. Presumption of causality. Reduce the burden of proof for those harmed. If harm follows AI system use and a plausible causal pathway exists, presume causation unless the developer or provider demonstrates otherwise. This shifts investigative burden to entities with better access to system internals.
2. Strict liability. Hold developers or deployers liable for harm regardless of fault or negligence. This approach treats AI-enabled biological harm like product liability: if your system contributed to harm, you compensate those affected. The advantage is clear accountability; the risk is discouraging beneficial AI development.
3. No-fault compensation funds. Establish pooled funds (contributed by AI companies, governments, or insurers) to compensate victims without requiring proof of specific fault. This model parallels vaccine injury compensation programs and could ensure redress without protracted litigation.
These frameworks assume identifiable developers and deployers. Open-weight models complicate liability: once released, anyone can deploy them, and the original developer has no control over downstream use. Strict liability for open-source developers could end open-source AI development; no liability leaves victims without recourse. This tension remains unresolved.
Current State
No jurisdiction has established comprehensive liability rules for AI-enabled biological harm. The EU AI Act addresses some liability questions for high-risk AI systems but does not specifically cover biosecurity applications. US product liability law could apply if AI systems are treated as products, but case law is limited.
Liability frameworks will likely develop reactively, after an incident clarifies the stakes. Proactive engagement now could shape more balanced rules before a crisis forces hasty legislation.
Recommendations for Different Actors
For Policymakers
- Designate lead responsibility for AI-bio governance within government
- Strengthen DNA synthesis screening by moving from voluntary to mandatory frameworks
- Fund biosecurity evaluation of AI systems through existing research mechanisms
- Engage internationally through existing forums (BWC, G7/G20) and new mechanisms
- Build technical capacity in regulatory agencies to evaluate AI-bio risks
For AI Companies
- Implement structured biosecurity evaluation before deploying models with biological capabilities
- Share threat information with government and peer companies
- Support development of industry standards through consortium action
- Invest in defensive applications of AI for biosecurity
- Be transparent about capabilities and limitations of internal governance
For Researchers
- Engage with biosecurity implications of biological AI research
- Practice responsible publication for dual-use findings
- Participate in evaluation efforts to build collective understanding
- Inform policy through accessible communication of technical issues
- Build cross-disciplinary bridges between AI safety and biosecurity communities
For International Bodies
- Convene multi-stakeholder dialogues on AI-bio governance
- Develop technical guidance that can be adapted to national contexts
- Build monitoring capacity for emerging biological technologies
- Facilitate information sharing across jurisdictions
- Document emerging norms and best practices
Why don’t existing biosecurity frameworks like the BWC cover AI risks?
Traditional biosecurity frameworks were designed for state programs, physical pathogens, and traditional laboratories. They regulate possession of dangerous materials but not information or AI-generated designs. The BWC lacks verification mechanisms and AI provisions, select agent regulations don’t cover computational outputs, and DURC policies focus on federally funded research rather than AI systems.
What is compute governance and how does it address AI-bio risks?
Compute governance monitors large-scale computational resources used to train powerful AI models. It includes reporting requirements above certain thresholds, know-your-customer standards for cloud providers, and export controls on advanced chips. However, biological design may not require frontier-scale compute, limiting its effectiveness specifically for AI-bio risks.
What is layered defense in biosecurity policy?
Layered defense uses multiple overlapping controls across different intervention points: restricting AI capabilities, controlling use-time through monitoring, blocking materials access via DNA synthesis screening, and enabling detection and response. No single layer is sufficient, but together they create barriers requiring adversaries to evade multiple controls.
Should AI models for biological design be open-source?
This remains contested. Open weights enable anyone to download and modify models, stripping out safety guardrails through jailbreaking. However, open science drives innovation and prevents monopoly control. The biosecurity concern is that once released, high-capability biological design tools cannot be un-released, requiring different governance than typical open science.
Who is liable when AI enables biological harm?
No jurisdiction has established clear liability rules for AI-enabled biological harm. Responsibility is distributed across the AI value chain: developers who build models, providers who integrate them, and deployers who use them. WHO proposes three approaches: presumption of causality (shift burden of proof to developers), strict liability (compensate regardless of fault), and no-fault compensation funds (pooled resources for victims). Open-weight models further complicate liability since anyone can deploy them without the original developer’s control.
This chapter is part of The Biosecurity Handbook. For the governance frameworks for AI-bio convergence, see also the chapters on DNA Synthesis Screening, the BWC and International Governance, and Red-Teaming AI Systems.