LLMs and Information Hazards
In 2023, MIT Media Lab researchers tasked chatbots with assisting with hypothetical pandemic-pathogen planning (Soice et al., 2023, preprint). Within one hour, the LLMs suggested four potential pandemic pathogens, explained reverse genetics protocols, supplied names of DNA synthesis companies unlikely to screen orders, and recommended contract research organizations for those lacking laboratory skills. RAND’s controlled experiments the following year found no statistically significant difference in biological attack plan viability between groups with LLM access and those using only internet search. Later studies produced a more differentiated result: substantial uplift on bounded digital biology tasks, but no significant increase in complete physical laboratory workflow success (Zhang et al., 2026, preprint; Hong et al., 2026, preprint).
- Define information hazards in the context of biosecurity and dual-use research, including the Bostrom typology.
- Evaluate LLM capabilities and limitations for providing actionable biological attack guidance.
- Analyze methodologies used in biosecurity evaluations (RAND, OpenAI, Anthropic, Gryphon).
- Distinguish between theoretical information access and operational attack capability.
- Explain why hallucinations are an unreliable barrier and why evaluations must measure correctness, not only whether a model responds.
- Assess safety-behavior post-training, refusals, and use-time safeguards as separate layers.
- Apply red-teaming concepts to evaluate AI systems for biosecurity risks.
This chapter discusses biosecurity risks at a conceptual level appropriate for education and policy analysis. Consistent with responsible information practices:
- Omitted: Actionable protocols, specific synthesis routes, exact pathogen sequences
- Included: Risk frameworks, governance mechanisms, policy recommendations
For detailed biosafety protocols, consult your Institutional Biosafety Committee and relevant regulatory guidance.
Introduction: Information Hazards in the AI Era
The concept of information hazards - information that could enable harm simply by being known - has long concerned biosecurity practitioners. The 1975 Asilomar Conference grappled with whether certain recombinant DNA techniques should be published. The 2011-2012 H5N1 gain-of-function controversy raised questions about whether transmissibility-enhancing mutations should be disclosed.
Large Language Models introduce a new dimension to this challenge. Unlike journal articles read by specialists, LLMs can synthesize information from across the scientific literature and present it in accessible formats to anyone who asks. They can answer follow-up questions, provide troubleshooting guidance, and adjust explanations to different expertise levels.
The question is not whether LLMs could provide harmful biological information - they can access dual-use knowledge from their training data. The questions are:
- Does LLM access meaningfully increase attack capability beyond existing resources?
- Can safety measures effectively prevent misuse while preserving beneficial uses?
- How should we evaluate and govern these systems as capabilities advance?
The empirical evidence on these questions reveals what we know, what remains uncertain, and what practitioners should understand about LLM information hazards.
Defining Information Hazards: The Bostrom Typology
The philosopher Nick Bostrom formally defined an information hazard as “a risk that arises from the dissemination of (true) information” that may cause harm or enable some agent to cause harm. This framing is essential: information hazards involve true information - not misinformation or deception.
In biosecurity, information hazards can be categorized by what they reveal:
Blueprint Hazards: Specific instructions enabling weapon creation. Examples include step-by-step protocols to synthesize poliovirus from mail-order DNA, or detailed procedures for weaponizing anthrax spores. These are the most direct form of information hazard.
Idea Hazards: General concepts that point adversaries in dangerous directions. For instance, suggesting “Have you considered using Variola minor instead of major?” or explaining that certain attenuated vaccine strains could be back-mutated to virulence. The specific protocol is not provided, but the conceptual direction is.
Signal Hazards: Information revealing that something is possible. The 2018 horsepox synthesis paper demonstrated that orthopoxviruses could be synthesized from commercially available DNA, even though the specific virus synthesized (horsepox) was not itself dangerous. The signal - “this class of viruses can be made from scratch” - was the hazard.
The primary demonstrated concern is not that LLMs invent new biology - current systems generally recombine or explain information in their training and tool contexts. The concern is that they can lower the search cost for existing dangerous information.
LLMs can aggregate dual-use research scattered across thousands of papers, protocols, and databases. Information that was technically public but practically obscure becomes accessible through natural language queries. This aggregation capability is what distinguishes LLM information hazards from simple internet search.
What LLMs Can and Cannot Do
Capabilities
Based on publicly available evaluations and disclosed capabilities, current frontier LLMs can:
Synthesize Existing Knowledge: - Explain complex biological concepts (pathogen biology, immune evasion, transmission dynamics) - Describe laboratory protocols for working with dangerous agents - Summarize the dual-use research literature - Provide general troubleshooting guidance for molecular biology techniques
Answer Stepwise Questions: - Break down complex procedures into sequential steps - Clarify ambiguous instructions - Adapt explanations to stated expertise levels - Suggest alternative approaches when asked
Access Specific Information: - Name pathogens with pandemic potential - Describe known enhancement mutations - Explain DNA synthesis and assembly methods - Discuss historical bioweapons programs
Limitations
Current LLMs cannot:
Generate Novel Attack Methodologies: - They synthesize from training data, not from first principles - Novel pathogen designs require specialized biological design tools (BDTs), not text models - Genuinely new attack vectors are not found in published literature
Provide Operational Specifics: - Exact synthesis routes for regulated toxins typically are not in training data - Supplier names, order procedures, and evasion techniques have limited coverage - Real-time information (current regulations, screening practices) is often outdated.
Bridge the Tacit Knowledge Gap (for now): - Text descriptions cannot substitute for hands-on laboratory training - Troubleshooting real experiments requires physical observation - Equipment operation, technique execution, and quality assessment require practice - However: Multimodal systems may provide visual or interactive assistance. Whether that assistance transfers into reliable wet-lab competence remains an open empirical question.
Guarantee Accuracy: - Hallucinations are common in technical domains - Safety-critical details may be confidently stated but incorrect - Following LLM protocols without expertise could result in failed (or dangerous) experiments
LLMs can confidently provide incorrect information. In biosecurity contexts, this is a double-edged sword: errors may impede some naive misuse, but hallucinations are not an unintentional safety feature that defenders can rely on.
For would-be attackers: Incorrect protocols can cause failures, but a correct answer can still be harmful. LLMs can state technically impossible or unsupported procedures with confidence. For a novice without the expertise to evaluate claims, following hallucinated protocols is a trap, but error is not a dependable safeguard.
For defenders: We cannot assume LLM errors provide reliable protection. Some outputs will be accurate, and sophisticated actors can cross-check accuracy against primary sources. Hallucination rates vary by domain - models are more reliable when information is well-represented in training data.
For evaluation: Studies measuring “information access” must distinguish between any response and correct response. An LLM that provides a detailed but wrong protocol has provided information without providing capability.
As one analysis noted, LLMs “simulate reasoning through statistical correlation, not symbolic operations” and “remain unsuitable for tasks requiring stateful logical chains.” For bioweapons development, where protocols must be followed exactly and errors can be fatal to the perpetrator, this imprecision matters.
Hallucination is not the only failure mode that matters in dual-use settings. The opposite failure mode, unwarranted affirmation, can also be dangerous. A 2026 Science study across 11 leading AI models found that they affirmed users’ actions 49% more often than humans, including in scenarios involving deception, illegality, or other harms (Cheng et al., 2026). Modeling work further suggests that repeated validation can drive “delusional spiraling” even in non-adversarial settings (Chandra et al., 2026, preprint).
In biosecurity, the concern is not that sycophancy teaches new biology by itself. It is that it can reinforce malicious framing, encourage persistence after setbacks, and reduce the chance that a model challenges dangerous intent. Evaluation suites should therefore test not only factual accuracy and refusal rates, but also whether models resist validating grievance narratives, evasion rationales, and harmful mission framing.
The Empirical Evidence: Uplift Studies
Defining “Uplift”
How do we measure whether an LLM is actually dangerous? The key metric is uplift - the marginal advantage an adversary gains by using AI compared to using standard tools (Google, Wikipedia, textbooks, scientific literature).
Conceptually:
Uplift = Capability(Human + AI) − Capability(Human + Internet)
If the uplift is zero, the AI is not adding biosecurity risk - even if it answers dangerous questions - because the user could have found equivalent information through conventional research. The biosecurity-relevant question is never “Can an LLM provide this information?” but rather “Does LLM access meaningfully improve attack capability beyond existing resources?”
This framing acknowledges that dangerous information already exists in various forms. The question is whether AI access makes that information more accessible, synthesized, or actionable in ways that matter operationally.
RAND Corporation Red Team Study (2024)
The RAND study remains the most rigorous public assessment of LLM impact on biological attack planning.
Methodology: - Recruited participants with varying backgrounds - Randomly assigned to “Internet only” or “Internet + LLM” conditions - Asked to develop biological attack plans - Expert panel evaluated plan viability
Key Finding: > “Current AI models do not meaningfully increase the risk of a large-scale biological weapons attack.”
The study found no statistically significant difference in plan viability between conditions. LLMs helped with brainstorming and information synthesis, but this did not translate to more viable attack plans.
Limitations: - Constrained timeframe (hours, not weeks) - Participants were not actual threat actors - Evaluated planning, not execution - Frontier models continue advancing
OpenAI Biosecurity Evaluation (2024)
OpenAI’s study assessed GPT-4 uplift across both expert and student cohorts, using tasks developed by Gryphon Scientific biosecurity specialists.
Methodology: - 100 participants: 50 biology experts (PhDs with professional wet lab experience) and 50 students (at least one university-level biology course) - Random assignment: control group (internet access only) vs. treatment group (internet plus GPT-4) - Five tasks, one per threat creation stage: Ideation, Acquisition, Magnification, Formulation, and Release - Five evaluation metrics per task: accuracy, completeness, innovation, time taken, self-rated difficulty
Key Finding: > GPT-4 provides “at most a mild uplift” in biosecurity-relevant tasks.
Accuracy uplift was 0.88 points (experts) and 0.25 points (students) on a 10-point scale; completeness uplift was 0.82 (experts) and 0.41 (students). Neither difference reached statistical significance. GPT-4’s primary value was saving time on literature search, not providing information unavailable through other means.
Stage-specific exception: For the Magnification and Formulation stages, AI assistance raised student performance to the expert baseline – students with GPT-4 reached the same level as experts without it. Aggregate uplift was non-significant, but these two stages represent a meaningful risk concentration even when overall results appear reassuring.
Notable Observation: Experts were better at extracting useful information from GPT-4 than novices, suggesting LLMs amplify existing expertise rather than substitute for it. This pattern has implications for threat modeling: the greatest near-term risk may not be enabling completely untrained actors but raising the ceiling for partially trained ones.
Anthropic Claude Evaluation (2024)
Anthropic’s Claude 3 Model Card disclosed biosecurity evaluation results.
Key Finding: Claude 3 models “substantially increased risk in certain parts of the bioweapons acquisition pathway” for novices but did not appear capable of uplifting experts “to a substantially concerning degree.”
Interpretation: The “acquisition pathway” includes many steps - identifying agents, planning approaches, obtaining materials, executing procedures. LLMs may help with early planning steps while providing less value for later execution steps that require tacit knowledge and physical access.
Gryphon Scientific Assessment (2023-2024)
A Gryphon Scientific assessment for USAID and subsequent work with OpenAI took a more granular approach than other evaluations. Public descriptions do not provide enough methods and outcome detail to treat the often-cited troubleshooting comparisons below as independently reproducible estimates.
What can be concluded: The publicly available summaries indicate that the work examined task-level assistance, including troubleshooting, but they do not establish that current models perform at a postdoctoral level across laboratory problems.
Implication: Troubleshooting deserves explicit testing because it could matter even when aggregate attack-planning scores do not change. Any such claim should be tied to a reproducible task set, model version, and human baseline.
Public summaries of Gryphon’s work describe more assistance for early-stage planning than for material acquisition or laboratory execution, a pattern consistent with the tacit-knowledge framework. The underlying task-level results should be obtained before treating that pattern as a measured effect.
In Silico Novice Uplift: Zhang et al. (2026)
A 2026 preprint evaluated 57 participants classified as biology novices from self-reported backgrounds across eight digital biology benchmark suites. A mixed-effects analysis estimated higher odds of correct responses with access to multiple frontier LLMs than with internet-only access (odds ratio 4.16, 95% CI 2.63–6.87). Adjusted accuracy increased from approximately 5% to more than 17%, and LLM-assisted participants outperformed controls on seven of eight benchmark suites (Zhang et al., 2026, preprint).
The result is evidence of substantial uplift on bounded digital tasks, not physical attack capability. The study combined heterogeneous cohorts and assignment designs, model access changed during data collection, some benchmark questions were available online, and participants could not be blinded to LLM access. Standalone LLMs also often outperformed LLM-assisted participants. These limitations, together with the absence of physical laboratory outcomes, constrain generalization from benchmark performance to operational execution.
Physical-World Validation: Hong et al. RCT (2026)
Prior biosecurity evaluations measured knowledge, planning, or other digital-task outcomes rather than complete physical execution. The first preregistered, investigator-blinded, randomized controlled trial (n=153) to measure whether LLMs improve novice performance in physical laboratory tasks was published in 2026 (Hong et al., 2026, preprint).
Methodology: Novice participants were randomized to LLM-assisted or internet-only conditions. Tasks modeled a viral reverse genetics workflow. The primary endpoint was full workflow completion. The study was conducted June-August 2025 using mid-2025 frontier LLMs.
Key Findings:
- No significant difference in workflow completion (5.2% LLM vs. 6.6% internet, P=0.759)
- No significant difference in individual task success rates
- Numerically higher LLM success in 4 of 5 tasks, notably cell culture (68.8% vs. 55.3%, P=0.059)
- Bayesian modeling estimated approximately 1.4-fold increase in success for a typical reverse genetics task under LLM assistance (95% CrI 0.74–2.62)
- Ordinal regression showed LLM participants more likely to progress through intermediate steps (posterior probability 81%–96%)
Interpretation: Mid-2025 LLMs did not substantially increase novice completion of complex laboratory procedures but were associated with modest intermediate-step benefits. This preprint is not a validated estimate for every model or laboratory setting, and the gap between in-silico benchmark performance and real-world utility remains substantial.
Why this matters for policy: Digital benchmark performance does not establish complete physical execution. The Hong trial directly tested that translation and found no significant increase in full workflow completion, while leaving open the possibility of smaller or task-specific benefits.
Synthesis: What the Evidence Shows
| Attribute | Evidence |
|---|---|
| Planning assistance | Mixed results across earlier studies; effects depend on task and evaluation design |
| Digital novice uplift | Zhang preprint (n=57): OR 4.16 for correct responses across eight benchmark suites |
| Expert comparison | LLM-assisted novices exceeded internet-only expert baselines on three of four benchmarks with expert data in the Zhang preprint |
| Novel methodologies | Not established by current uplift studies |
| Operational guidance | Digital performance does not resolve tacit knowledge and physical access barriers |
| Physical-world validation | Hong RCT (n=153): no significant increase in full workflow completion; modest intermediate-step benefit |
| Trend direction | Capabilities advancing; assessments are snapshots in time |
The studies summarized above reflect specific model versions tested at specific points in time. “Mild uplift” with GPT-4 in 2024 does not guarantee “mild uplift” with GPT-5 or Claude 4. AI capabilities are advancing rapidly, and the marginal risk calculation could shift substantially with next-generation models. Policymakers should treat these findings as baselines for continuous monitoring, not as permanent assurances.
“No Significant Difference” Is Not the Same as “No Difference”
Several reassuring findings in the table above rest on null results: RAND found “no statistically significant difference” in plan viability; OpenAI found that neither accuracy nor completeness uplift “reached statistical significance” at n=100; Hong et al.’s RCT (n=153) found no significant difference in workflow completion (P=0.759). A 2024 statistical framework for evaluating AI systems makes a point that applies directly to this evidence base: failing to reach statistical significance in a modest sample is not proof that no meaningful effect exists, it can equally mean the study lacked the statistical power to detect an effect of the size that would actually matter for policy (Miller, 2024, preprint). The framework’s power-analysis formula shows that resolving a moderate effect with conventional error rates typically requires several hundred to well over a thousand independent observations, well above the sample sizes available in most published uplift studies.
The Hong et al. numbers illustrate why this distinction is not academic. Cell culture success was 68.8 percent with LLM assistance versus 55.3 percent without (P=0.059), a 13.5 percentage-point gap that narrowly missed the conventional 0.05 threshold in a study of 153 participants. Whether that reflects a true absence of uplift or a study underpowered to confirm a real double-digit effect is exactly the question a preregistered power calculation should answer before data collection, not a question left to post hoc interpretation of a borderline p-value. The same caution applies to OpenAI’s five-task, five-metric design: when several related outcomes are measured on the same participants, the comparisons are correlated rather than independent, and a framework built for independent significance tests can understate the true uncertainty across that correlated set.
Why this matters for governance: capability thresholds that gate model deployment or trigger safeguards are ultimately built on evaluations like these. A policy conclusion of “no meaningful uplift” is only as strong as the statistical power behind it. Evaluations used to inform binding capability thresholds should report the effect size the study was powered to detect, not only whether the observed effect crossed a significance threshold.
Safety Measures and Their Limitations
Safety-Behavior Post-Training
Frontier systems use varied post-training methods to shape refusals and other safety behavior. RLHF is one preference-based method, while other pipelines use AI feedback, safety demonstrations, critiques, or explicit principles. These methods influence model disposition, but do not replace use-time classifiers, permissions, access controls, or monitoring. See Post-Training and Deployment Safeguards for the canonical distinction.
When asked for harmful assistance, a post-trained model may:
- Decline to provide assistance
- Explain why the request is problematic
- Suggest legitimate alternatives (research, education)
Anthropic’s Constitutional AI preprint describes an approach using explicit principles and AI feedback. It is one implementation, not evidence that principle-based post-training or any other single layer prevents adversarial use.
The Jailbreaking Problem
Safety training can be bypassed. Research on adversarial prompts demonstrates that aligned models can be manipulated to produce harmful outputs through:
- Role-playing scenarios: “As a fiction writer researching…”
- Hypotheticals: “For educational purposes, explain how one would…”
- Token manipulation: Character-level tricks to bypass content filters
- Multi-turn extraction: Building context across many messages
Implications for Biosecurity:
- Evaluations should separate layers: Measure capability under defined elicitation conditions, model disposition, and the complete deployed system.
- Jailbreaks proliferate: Once discovered, they spread through online communities
- Cat-and-mouse dynamics: Safety teams patch jailbreaks; new ones emerge
- Open-source models: Local deployment may remove API-level safeguards entirely
For biosecurity evaluation, we care about:
Capability: What can the model do under defined elicitation, tool, and resource conditions?
Disposition: What does the post-trained model tend to do under its current instructions?
Use-time safeguards: What do classifiers, permissions, access controls, monitoring, and other controls change in the configured system?
Post-training can add, reduce, mask, or retain measured capabilities, while use-time safeguards can change which behavior is accessible. No one layer establishes the safety of the others.
The Anthropic Responsible Scaling Policy framework addressed this by tying safety requirements to capability thresholds, not just deployed behavior. In February 2026, Anthropic revised the RSP to remove the binding deployment pause commitment that had been its core enforcement mechanism, a development with significant implications for biosecurity governance. See Responsible Scaling Policies for analysis.
Content Filtering and Monitoring
Major LLM providers implement additional safeguards:
- Input filtering: Detect and block suspicious queries
- Output filtering: Screen responses for dangerous content
- Usage monitoring: Flag accounts with concerning patterns
- Rate limiting: Prevent systematic extraction attempts
These measures reduce casual misuse but are unlikely to stop determined, sophisticated actors who can use multiple accounts, obfuscated queries, or open-source models.
Constitutional Classifiers
Constitutional Classifiers, developed by Anthropic, are a production use-time defense against jailbreaks. These systems use classifiers trained on synthetic data to monitor exchanges in real time. The January 2026 generation evaluates model outputs in the context of their inputs, addressing reconstruction attacks that fragment harmful requests and output obfuscation that disguises responses.
Anthropic reported that its initial red-teaming campaign found no universal jailbreak, but the UK AI Security Institute subsequently developed an automated black-box method that produced universal jailbreaks against the system (UK AI Security Institute, February 2026). SecureBio’s July 2026 external review also found that high default refusal coverage did not establish adversarial robustness and that highly skilled actors could still make substantial progress (SecureBio, July 2026). Classifier coverage, jailbreak resistance, exempt-user monitoring, and remediation therefore require separate evidence. See Red-Teaming AI Systems for Biosecurity Risks for the full evaluation framework.
Policy Architecture: Hardcoded vs. Instructable Behaviors
Beyond training techniques, frontier model developers have published explicit policy architectures governing what operators and users can and cannot modify. OpenAI’s Model Spec (December 2025) establishes CBRN safety as a root-level commitment that “cannot be overridden by system messages, developers or users.”
What is prohibited at root level:
- “Steps related to creating, obtaining, magnifying, or deploying chemical, biological, radiological, and/or nuclear (CBRN) weapons”
- No exception exists for “informational or historical questions” when detailed harmful steps are requested
What remains permitted:
- General factual information (e.g., confirming an agent is regulated under the Select Agent Program)
- Educational or historical context
- Discussion of dangers and legal implications
Operator limitation principle: An API customer cannot instruct a frontier model to provide CBRN uplift by claiming a legitimate use case. No system prompt overrides root-level safety behaviors. The Model Spec distinguishes hardcoded behaviors (stable regardless of deployment context) from instructable behaviors (adjustable by operators within defined bounds). CBRN prohibition is hardcoded. Beyond model behavior, automated monitoring flags potential violations for law enforcement referral when credible imminent threat signals are present.
This architecture governs API-deployed models only. Locally downloaded open-weight models operate without these constraints, and no model-level policy closes that gap. See The Jailbreaking Problem for the related issue of adversarial bypass.
Red-Teaming for Biosecurity
What Is Red-Teaming?
In security contexts, red-teaming involves adopting an adversarial mindset to identify vulnerabilities. For AI biosecurity, this means:
- Attempting to extract harmful biological information
- Testing whether safety measures can be bypassed
- Evaluating whether outputs would be operationally useful
- Simulating threat actor behavior and capabilities
Red-Teaming Methodologies
Expert Elicitation: Biosecurity experts attempt to use LLMs for harmful purposes, evaluating: - What information can be obtained? - How accurate and complete is it? - Would it advance an attack beyond public resources?
Structured Scenarios: Defined attack scenarios (specific pathogen, target, actor profile) guide evaluation: - Can the LLM provide relevant information for each attack stage? - Where do guardrails activate? - What gaps remain that other resources would fill?
Automated Probing: Systematic testing of model responses to biosecurity-relevant queries: - Gradient-based attacks to bypass safety training - Prompt injection techniques - Coverage mapping across pathogen/technique space
Challenges in Red-Teaming
Information hazard recursion: Documenting how to extract dangerous information creates another information hazard. Evaluation reports must balance transparency with security.
Expertise requirements: Effective red-teaming requires both AI/ML expertise and biosecurity domain knowledge - a rare combination.
Temporal limitations: Evaluations are snapshots. Model updates, new jailbreaks, and evolving techniques change the landscape continuously.
Generalization: Success or failure on specific test cases may not generalize to real threat scenarios.
For more on red-teaming methodologies, see the Red-Teaming AI Systems chapter later in Part IV.
The Information Hazard Landscape
Categories of Concern
Dual-Use Knowledge: - Enhancement mutations increasing transmissibility or virulence - Immune evasion modifications - Stability and environmental persistence optimization
Operational Information: - DNA synthesis providers and their screening practices - Acquisition routes for precursor materials - Evasion techniques for regulatory controls
Synthesis and Assembly: - Protocols for assembling dangerous pathogens from synthetic DNA - Reverse genetics systems for RNA viruses - Chimeric agent construction methods
Historical and Technical References: - Declassified bioweapons program documents - Technical manuals from state programs - Academic papers with dual-use content
What’s New vs. What’s Accessible
A critical question is whether LLMs provide access to information that was previously unavailable or merely make accessible information easier to find.
Genuinely restricted information: - Classified intelligence on state programs - Unpublished proprietary research - Controlled technical data (export-restricted)
LLMs cannot provide access to genuinely restricted information not in their training data.
Technically accessible but practically obscure information: - Information scattered across many sources - Content in foreign languages - Older literature not easily searchable
LLMs may make this information more accessible by combining material across sources.
Readily accessible information: - Published peer-reviewed literature - Government reports and guidelines - Textbooks and educational materials
LLMs primarily synthesize this category, information that was already accessible to determined individuals.
The Library Metaphor
A useful conceptual framework: LLMs function as librarians for an already-existing library, not as authors of new books. They can locate, synthesize, and explain information scattered across their training data, but they cannot create knowledge that does not exist in that corpus.
This distinction matters for risk assessment. The concern is not that LLMs will invent novel attack methodologies; current models lack that capability. The concern is that information requiring months of literature review becomes accessible through a few queries. The aggregate effect of reducing search costs across many potential actors may create marginal risk increases even when no individual query provides unprecedented information.
RAND’s finding that LLM outputs “generally mirrored information readily available on the internet” confirms this framing (Mouton et al., 2024). LLMs do not expand the library; they make it easier to navigate. For biosecurity, this means the primary risk is democratized access to existing dual-use knowledge, not AI-generated novelty.
This has policy implications. Restricting LLM access to biological information is unlikely to succeed when the underlying sources remain publicly available. More tractable interventions target the chokepoints where information must translate to physical capability: DNA synthesis screening, laboratory access controls, and material restrictions.
The calculus differs for biological AI models (BAIMs) trained on specialized pathogen data rather than general text. Unlike LLMs, where training data is the entire open internet, BAIMs draw from narrow, classifiable datasets where access controls are feasible. The Biosecurity Data Level (BDL) framework proposes tiered restrictions on pathogen datasets used for AI training, targeting capability formation upstream of model release (Bloomfield et al., 2026). For BAIMs, data governance may be more tractable than model-output restrictions. See Training Data Governance for the full framework.
The Unmasking Hazard
Beyond providing dual-use knowledge directly, LLMs pose a less-discussed risk: unmasking previously obscured information through data aggregation.
Kevin Esvelt and colleagues have highlighted that LLMs can synthesize information across millions of sources to reveal patterns that individual sources do not disclose. If a publication states "A patient in [City X] with [Genetic Mutation Y] contracted [Virus Z]," an LLM scanning social media, genealogy databases, and public records could potentially triangulate the patient’s identity.
For biosecurity, unmasking risks include:
- Vulnerability identification: Synthesizing public information to identify security gaps at specific facilities
- Researcher identification: Connecting publications to identify key personnel with specific expertise
- Supply chain mapping: Aggregating procurement data to map acquisition routes for regulated materials
- Pattern recognition: Connecting seemingly unrelated dual-use papers to identify dangerous research directions
This means we cannot rely on "security by obscurity" - the assumption that dangerous information is protected simply by being scattered. If the component pieces exist online, even in fragments, LLMs may be able to stitch them together.
We are currently in what might be called the "Google Plus" era of AI biosecurity risk. Current LLMs are essentially better search engines - they synthesize and present existing information more efficiently, but they do not yet possess the reasoning capabilities to design novel pathogens or the physical agency to synthesize them.
However, this era may be ending. In December 2025, OpenAI reported a GPT-5 wet-lab collaboration in which human scientists performed the physical work while the model proposed and refined experimental plans. The company reported a large efficiency improvement, but that result has not been independently replicated. Direct LLM connection to a cloud laboratory remains a governance question addressed in Cloud Labs and Automated Biology.
Demonstrated (supported by published evidence):
- LLM uplift is task-dependent: a 2026 preprint found substantial improvement on bounded digital biology tasks (Zhang et al., 2026, preprint), while a separate randomized trial found no significant increase in full physical workflow completion (Hong et al., 2026, preprint)
- LLMs can synthesize scattered dual-use information more efficiently than search engines
- Adversarial prompts have bypassed safeguards in some aligned-model systems (Zou et al., 2023, preprint)
- Information access alone is insufficient for operational capability
- Current evaluations measure information access, not attack success
Theoretical (plausible but not yet demonstrated):
- LLMs providing “significant uplift” with future model generations
- Multimodal AI substantially eroding tacit knowledge barriers
- LLMs connected to cloud labs enabling autonomous biological work
- “Unmasking” attacks successfully identifying operational vulnerabilities
Unknown (insufficient evidence to assess):
- Whether capability thresholds will be crossed with next-generation models
- How quickly the “Google Plus” era will end
- The effectiveness of safety measures against sophisticated adversaries
- Whether open-source models will reach concerning capability levels
These distinctions inform appropriate policy responses: demonstrated risks warrant immediate intervention, while theoretical risks require monitoring frameworks and contingency planning.
Governance and Policy Implications
Current Approaches
Voluntary Industry Commitments: Major AI labs have committed to biosecurity evaluations before deploying frontier models. The Frontier Model Forum provides a venue for sharing best practices.
Government Engagement: The UK AI Security Institute (renamed from AI Safety Institute in February 2025) and U.S. Center for AI Standards and Innovation (CAISI) (renamed from AI Safety Institute in June 2025) conduct independent evaluations. Executive Order 14110 (issued October 2023, revoked January 2025) established reporting requirements for models above certain compute thresholds. NIST AI 800-1 (January 2025) provides official U.S. guidance for managing misuse risks from dual-use foundation models, including a dedicated appendix on chemical and biological threats. See Policy Frameworks for AI-Bio Convergence for the full framework.
Research Community Norms: The biosecurity research community has developed norms around dual-use research of concern (DURC). Similar norms are developing for AI-biosecurity research.
Risk Quantification Frameworks: Recent work has begun translating capability evaluations into quantified risk estimates. A 2025 GovAI analysis estimated that a 10 percentage point increase in capable individuals could raise annual epidemic probability from 0.15% to 1.0%. Such frameworks help policymakers move beyond qualitative “uplift” language to inform specific resource allocation decisions.
Chemical Threats: A Distinct Evaluation Gap
Current industry frameworks treat biological evaluations as proxies for the combined bio/chemical risk category (OpenAI Preparedness Framework v2, April 2025). The OPCW Scientific Advisory Board’s Temporary Working Group on AI (SAB/REP/1/26, March 2026) independently confirmed that chemical-specific evaluation gaps remain unaddressed. Chemical threats have characteristics that biological threat models do not fully capture:
- Explicit procedural steps: Chemical synthesis involves more explicit step-by-step procedures with less dependence on tacit laboratory knowledge than biological agent production, lowering the practical barrier when AI fills informational gaps.
- Distinct regulatory regimes: The Chemical Weapons Convention (CWC) and its Schedule 1/2/3 precursor controls operate differently from the Biological Weapons Convention. Evasion strategies (finding unregulated analogs, alternative synthetic routes) are specific to the chemical regulatory architecture.
- Novel compound generation: AI retrosynthesis tools can generate pathways for novel toxic compounds not yet represented in chemical weapons databases or covered by existing detection signatures. Known agents have established detection protocols and partial antidotes; novel compounds optimized for the same biochemical endpoint (e.g., acetylcholinesterase inhibition) while evading existing signatures represent a qualitatively different threat. The MegaSyn experiment demonstrated the mechanism: an inverted drug-discovery model produced VX analogs and novel structures with comparable predicted toxicity in under six hours (see AI as a Biosecurity Risk Amplifier).
Stage-based policy framework. Effective LLM chemical safety policy requires distinguishing threat creation stages, adapted from the five-stage biological threat creation model:
| Stage | Refuse | Permit | Boundary |
|---|---|---|---|
| Ideation | Agent selection by weapons profile | Toxicology education; occupational safety | Allow mechanism explanations; refuse target-selection guidance |
| Acquisition | Non-scheduled precursor substitutes for CWC routes | General procurement; regulatory compliance | Allow compliance queries; refuse circumvention queries |
| Synthesis | Routes for CWC Schedule 1/2 agents or analogs; yield/purity optimization for toxic targets | Named reactions; retrosynthesis for non-toxic targets | Allow reaction mechanisms; refuse application to toxic endpoints |
| Weaponization | Dispersal design; aerosolization of toxic agents | Aerosol science for inhalers; agricultural spraying | Allow general engineering; refuse when linked to harm maximization |
| Deployment | Target selection; detection evasion; transport concealment | Emergency response; decontamination; forensic methods | Allow defensive content; refuse offensive planning |
CWC Schedule structure directly defines the Acquisition and Synthesis stage boundaries in the table above (OPCW Annex on Chemicals):
| Schedule | Regulatory standard | Control point |
|---|---|---|
| 1 | “Little or no use for purposes not prohibited under this Convention” | Strictest controls; precursors controlled at the final synthesis stage |
| 2 | “Not produced in large commercial quantities for purposes not prohibited” | Intermediate controls; precursors at formation stage |
| 3 | “May be produced in large commercial quantities for purposes not prohibited” | Oversight reporting; widely available commercially |
AI retrosynthesis tools that identify Schedule 3 or non-scheduled starting materials for reaching Schedule 1 endpoints represent the core evasion risk: routes that bypass the regulatory architecture entirely, using commercially available reagents to synthesize agents subject to the strictest controls.
Two failure modes require equal attention. Under-refusal (providing actionable uplift) is the obvious concern. Over-refusal is the less-discussed failure: a policy that refuses “How does sarin inhibit acetylcholinesterase?” imposes real costs on chemistry students, toxicologists, emergency responders, and policy researchers without security benefit. The boundary is actionable uplift, not topic proximity.
Multi-turn progressive disclosure presents a distinct challenge that single-turn classifiers miss. A sequence of individually benign queries (precursor sourcing; reaction conditions; purification; dispersal) can collectively constitute a threat creation roadmap. Session-level monitoring across sequential turns, not just per-query evaluation, is required to detect this pattern.
Open Questions
Threshold setting: At what capability level should deployment require additional safeguards or restrictions?
Evaluation standardization: How can we create reproducible, comparable biosecurity evaluations across models and labs? The Center for AI Safety’s WMDP-Bio benchmark (1,273 biosecurity-specific questions) is one standardization effort, alongside the Virology Capabilities Test, a preprint benchmark of 322 multimodal virology questions in which the reported best model outperformed 94% of expert participants under the paper’s test conditions. Whether these benchmarks adequately capture operational risk (rather than knowledge access) remains debated.
Open-source governance: How should openly released models - which cannot be centrally controlled - be governed?
International coordination: How do we prevent regulatory arbitrage where dangerous models are developed in permissive jurisdictions?
Recommendations for Practitioners
Calibrate concern by outcome: Digital-task uplift and physical execution are different endpoints. Evaluation results should not be generalized beyond the task, model access, participant population, and outcome measured.
Focus on chokepoints: DNA synthesis screening, laboratory access controls, and material restrictions remain more impactful than AI-specific interventions.
Engage with evaluations: If you have biosecurity expertise, consider participating in red-team exercises and evaluations from AI labs and government agencies.
Monitor trends: This is a rapidly evolving field. Today’s assessment may not hold in 1-2 years as capabilities advance.
Support responsible disclosure: Report biosecurity vulnerabilities in AI systems through appropriate channels rather than publicizing them.
AI-enabled pathogen design poses risks beyond information access, with biological design tools (BDTs) creating new threat vectors (see AI-Enabled Pathogen Design). For the defensive side of the equation, covering biosurveillance, countermeasures, and novel threat detection, see AI for Biosecurity Defense.
What are information hazards in biosecurity contexts?
Information hazards are true information that could enable harmful actions when disseminated, even when generated without malicious intent. In biosecurity, this includes pathogen enhancement techniques, synthesis routes, and operational attack planning accessible through dual-use research literature. Nick Bostrom’s typology distinguishes blueprint hazards (specific instructions), idea hazards (dangerous conceptual directions), and signal hazards (revealing something is possible).
Do LLMs provide significant uplift for biological attacks?
Evidence is task-dependent. RAND 2024 found no statistically significant difference in attack plan viability, and OpenAI 2024 reported at most mild uplift with GPT-4. A 2026 preprint found substantially higher novice performance across bounded digital biology tasks (Zhang et al., 2026, preprint), while a separate 2026 randomized trial found no significant increase in full physical workflow completion (Hong et al., 2026, preprint). Digital uplift therefore should not be treated as proof that novices can execute complex laboratory work.
Can post-training and deployment safeguards be bypassed?
Yes. Research on adversarial prompts demonstrates that aligned models can be manipulated to produce harmful outputs. Safety-behavior post-training, model disposition, and use-time safeguards should therefore be evaluated separately from base capability and from one another. A successful refusal test does not establish that the underlying capability is absent, and a capability test does not establish how the deployed system behaves under its actual controls.
Why do AI hallucinations matter for biosecurity evaluation?
Hallucinations may cause some incorrect procedures to fail, but sophisticated actors can cross-check accuracy against primary sources. They therefore provide no reliable protection. Evaluation studies must distinguish between “any response” and “correct response” when measuring information access versus actual capability provision.
This chapter is part of The Biosecurity Handbook. For foundational context, see the previous chapter on AI as a Biosecurity Risk Amplifier. For connections to broader governance, see International Governance and the BWC and Dual-Use Research of Concern.