cs-552-2026-Clanker-Scientists/safeguard-deberta-ragtruth-v1

Safeguard — DeBERTa-v3-large, RAGTruth fine-tuned (faithfulness verifier)

DeBERTa-v3-large fine-tuned on RAGTruth for generated-answer faithfulness. 2-class head: 0 = supported, 1 = hallucinated. Used as the pipeline's grounding verifier: premise = cited context, hypothesis = drafted answer, argmax == 0 -> supported.

M3 results

dataset	macro-F1
RAGTruth (in-domain)	0.80 (hallucination P/R 0.71 / 0.79)
CogniBench (transfer)	0.66
MEMERAG-FR (transfer)	0.58

The earlier ContractNLI proxy checkpoint scored 0.838 on its proxy but only 0.52 zero-shot on real RAGTruth faithfulness — use this checkpoint for grounding generated answers.

from transformers import AutoModelForSequenceClassification, AutoTokenizer
tok = AutoTokenizer.from_pretrained("cs-552-2026-Clanker-Scientists/safeguard-deberta-ragtruth-v1")
mdl = AutoModelForSequenceClassification.from_pretrained("cs-552-2026-Clanker-Scientists/safeguard-deberta-ragtruth-v1")
# supported iff mdl(**tok(context, answer, return_tensors="pt")).logits.argmax(-1) == 0