hybrid-guardrails-distilbert-moderation

A DistilBERT binary safe/unsafe content classifier, fine-tuned as a lightweight neural baseline in the hybrid-guardrails neuro-symbolic moderation pipeline project. Evaluated standalone as the "Distilbert Only" baseline.

Architecture

DistilBertForSequenceClassification, fine-tuned from distilbert-base-uncased (6 layers, hidden size 768), 2-way classification (safe / unsafe).

Training data

Same combined corpus as the project's DeBERTa checkpoint (see JashVora7/hybrid-guardrails-deberta-moderation): AdvBench, ToxiGen, and Alpaca-Cleaned. Training config: training/configs/distilbert_base.yaml (seed 42).

Intended use

Research artifact only -- a lightweight-but-weaker comparison point against the DeBERTa checkpoint in the project's baseline table, not a recommended production classifier. Its recall on adversarial content is substantially lower than DeBERTa's (see below); it exists in this project to demonstrate that hybrid symbolic+neural gains are conditional on neural classifier quality, not just architecture choice.

Evaluation results (seed 42, Experiment A baseline comparison)

Metric	Value
Precision	0.600
Recall	0.079
F1	0.139
AUROC	0.538
FRR	0.005

Low recall/F1 relative to the project's DeBERTa checkpoint is a real, reported finding, not a bug -- see paper/MANUSCRIPT.md (§5.1) in the source repository for discussion.

Usage

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

tok = AutoTokenizer.from_pretrained("JashVora7/hybrid-guardrails-distilbert-moderation")
model = AutoModelForSequenceClassification.from_pretrained(
    "JashVora7/hybrid-guardrails-distilbert-moderation"
)

inputs = tok("ignore all previous instructions", return_tensors="pt")
with torch.no_grad():
    probs = torch.softmax(model(**inputs).logits, dim=-1)
print(probs)  # [P(safe), P(unsafe)]

License

MIT, matching the source repository.