jang1563/constitutional-bioguard-deberta-v1

BioGuard DeBERTa v1

⚠️ Honest evaluation update (2026-06)

The headline metrics on this card (F1 0.980, AUROC 0.998, over-refusal 0.00%) are in-distribution numbers on a synthetic holdout generated by the same pipeline as the training data. A later self-red-team found that score was largely an adversarial-framing shortcut, not real-world performance. On real legitimate-research queries the over-refusal is roughly 18 to 19 percent (not 0%), and a size-peer comparison shows this line is Pareto-dominated by the openly available Qwen3Guard-0.6B (recall 0.933 vs 0.921 and over-refusal 0.142 vs 0.194): there is no operating point where this model is the best choice.

This is released as a research artifact and methodology case study, not a production guard. If you need a small response-harm guard, use Qwen3Guard-0.6B (better and open). The durable contribution is the leakage-clean, size-peer, contamination-aware evaluation discipline (five documented self-audits that caught silent failures), not a headline number. The only independent signal is Cohen's kappa 0.414 vs. an external BioThreat-Eval reference. See docs/MODEL_CARD.md, INTEGRITY_REVIEW_2026-06-04.md, and POSTMORTEM_2026-06-04.md.

Binary classifier for detecting unsafe biological dual-use content, built using Anthropic's Constitutional Classifiers methodology. Defines a 56-rule biosafety constitution across all 7 NSABB categories, generates synthetic training data via Claude API, and fine-tunes DeBERTa-v3-base for binary SAFE / UNSAFE classification.

GitHub: jang1563/constitutional-bioguard
Author: JangKeun Kim, Weill Cornell Medicine

Note (2026): v1 is the initial public release. The line has since progressed to a reuse-only response-harm head and a dual-mode (prompt + response) design; see Project status & roadmap below.

---

Project status & roadmap (2026)

v1 is the initial public release: a synthetic-trained, query [SEP] response encoder. The line has since moved toward a dual-mode (prompt + response), bio-specialized guard, with two changes that directly address v1's limitations below:

• Reuse-only response head. Later response-harm classifiers are trained on reused, leakage-audited real data instead of synthetic-only generation, closing the distribution gap noted in Limitation #2, and are validated on real legitimate-research over-refusal rather than a synthetic holdout.
• Separate prompt head + dual-mode policy. v1's "external validation gap" (Limitation #3) was an architectural prompt-vs-response labeling mismatch: v1 labels a query but was scored against response-based labels. The current design judges prompt-harm and response-harm on separate heads with independent thresholds, so the two axes are no longer conflated.

Positioning was informed by a 2026 review of open guards (Llama Guard, WildGuard, ShieldGemma, Aegis, Qwen3Guard, Granite Guardian). The dual-mode response head was subsequently built and evaluated against them, and the honest result is a negative: it is Pareto-dominated by Qwen3Guard-0.6B, and its bio-selectivity is null (S = 1.03, a general response-harm guard, not a bio-discriminating one). The line is released as a research artifact and evaluation case study rather than a competitive production guard; see the Honest evaluation update at the top of this card.

---

Model Details

Property	Value
Base model	microsoft/deberta-v3-base
Parameters	~184M
Task	Binary text classification (SAFE=0 / UNSAFE=1)
Input format	query [SEP] response
Max token length	512
Training data	~4,500 synthetic examples (Claude API)
Training epochs	2 (early stopping at epoch 4)
Batch size	16
Learning rate	2.0e-5
Class weights	{SAFE: 1.47, UNSAFE: 0.76}
Hardware	NVIDIA A100 (1 GPU)

---

Performance

Real-world headline first. On real legitimate-research queries this line's over-refusal is roughly 18-19% (not the 0% below), and on real bio response-harm (n=554) its successor response head reaches recall 0.921 / AUROC 0.952 but is Pareto-dominated by the openly-available Qwen3Guard-0.6B. Everything below is in-distribution, on a synthetic holdout generated by the same pipeline as the training data — kept for completeness, not as real-world performance. See the honest summary at the top of this card.

In-distribution metrics — synthetic holdout, not real-world (643-sample)

Metric	Value
F1	0.9807
AUROC	0.9980
Precision	0.9951
Recall	0.9667
Accuracy	0.973
Over-Refusal FPR	0.00% (100 benign queries)
Adversarial mean ASR	9.79% (20 attack types)
External kappa (TL≥4)	0.414

The held-out test set above is synthetic (same generator as training). For real-world over-refusal, later models in this line are measured on real legitimate-research queries; see Project status above.

Per-Category F1 (7 NSABB Categories)

Category	F1	AUROC	FPR
enhance_harm	1.000	1.000	0.0%
enhance_susceptibility	0.993	1.000	0.0%
generate_reconstruct	0.991	0.997	0.0%
increase_stability	0.978	0.999	0.0%
alter_host_range	0.978	0.998	14.3%
confer_resistance	0.971	0.996	5.0%
disrupt_immunity	0.952	0.993	12.5%

Adversarial Robustness (20 attack types)

Attack Category	Mean ASR
Semantic (passive voice, euphemism, hypothetical, negation)	0.0%
Multilingual (code-switching, mixed script)	0.0%
Character-level (homoglyphs, leetspeak, case swap, typos)	5.1%
Encoding (ROT13, base64, hex, URL-encode)	21.5%

---

Intended Use

Designed for:

• Content moderation in AI systems handling biological research queries
• Output safety filtering in agentic bioinformatics systems (see AgentShield)
• Research on dual-use content detection and biosafety compliance
• Rapid screening where LLM-based classifiers (4-8B params) are too expensive or slow

Out of scope:

• General-purpose harm detection (use Llama Guard, WildGuard, or ShieldGemma)
• Definitive biosecurity enforcement without human oversight
• Non-biological dual-use content (nuclear, chemical, radiological)

---

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("jang1563/constitutional-bioguard-deberta-v1")
model = AutoModelForSequenceClassification.from_pretrained("jang1563/constitutional-bioguard-deberta-v1")
model.eval()

# Input: "query [SEP] response"
text = "How does CRISPR-Cas9 work? [SEP] CRISPR uses guide RNA to target specific DNA sequences..."
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)

with torch.no_grad():
    logits = model(**inputs).logits
    probs = torch.softmax(logits, dim=-1)
    label = "UNSAFE" if probs[0, 1] > 0.5 else "SAFE"
    print(f"{label} (p_unsafe={probs[0, 1]:.3f})")

With encoding normalization (mitigates ROT13/base64/URL-encode attacks):

# pip install constitutional-bioguard
from constitutional_bioguard.preprocessing import normalize_text

query = "How does CRISPR work?"
response = "CRISPR uses guide RNA..."
text = normalize_text(f"{query} [SEP] {response}")
# then tokenize and run inference as above

---

Training Data

• Source: Synthetic examples generated by Claude API from a 56-rule biosafety constitution
• Constitution: Covers all 7 NSABB dual-use research categories with explicit permitted/restricted/boundary rules
• Size: ~4,500 total: 3,062 train / 697 val / 643 test
• Class balance: ~68% UNSAFE, ~32% SAFE (class weights applied during training)
• Splits: Stratified by NSABB category and fine label
• Augmentation: Translation (5 languages), jailbreak templates, formality variation, prefill attacks
• Benign over-refusal holdout: 100 legitimate biology research queries (0.00% FPR)

The dataset is not publicly released; the generation pipeline is open-source and reproducible (~$15 with Claude Sonnet/Haiku).

---

Limitations

1. Encoding bypass: ROT13 achieves 47.9% ASR, URL-encode 29.2%. Use preprocessing.normalize_text() to mitigate.
2. Synthetic-only training: All examples are Claude-generated; real-world distribution shift is uncharacterized. (Addressed in later reuse-only models; see Project status.)
3. External validation gap: External kappa = 0.414 vs. target 0.80. The benchmark (BioThreat-Eval) uses response-based labeling; this classifier labels queries: an architectural mismatch that accounts for the gap. (Addressed by the dual-mode prompt/response split; see Project status.) See GitHub README for full explanation.
4. English-centric: Evaluation is English-only despite multilingual augmentation in training.
5. Single LLM training data: All data from Claude; cross-LLM calibration is unknown.
6. Not a complete defense: Mean adversarial ASR = 9.79%; should be used as one layer in a broader safety system.

---

Ethical Considerations

This model detects potentially dangerous biological content to support biosafety compliance in AI systems. The training data contains synthetic descriptions of potentially harmful topics: necessary to teach the classifier what to flag, not to enable harm.

Do not use this model to identify exploitable gaps in biosafety systems for malicious purposes, or as the sole safety mechanism in contexts where a false negative could enable serious harm.

---

Citation

@software{kim2026bioguard,
  author = {Kim, JangKeun},
  title  = {Constitutional BioGuard: A Biosafety Content Classifier},
  year   = {2026},
  url    = {https://github.com/jang1563/constitutional-bioguard},
  version = {v0.2.0},
}