phuy1125/xlmr-base-fakenews-kd

XLM-RoBERTa-base — Cross-lingual Fake-News Detection (KD student)

Student model distilled from an XLM-RoBERTa-Large teacher for cross-lingual (EN → VI) fake-news / claim verification. Trained with Knowledge Distillation (soft + hard loss, α=0.7, temperature T≈1.094).

• Backbone: xlm-roberta-base
• Params: 278.2M
• Labels: 0 = fake (tin giả), 1 = real (tin thật)

Results (best seed = 42)

Split	Score
VI test (F1-macro)	0.8401

• Retention vs teacher: 94.8%

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

REPO = "phuy1125/xlmr-base-fakenews-kd"
tok = AutoTokenizer.from_pretrained(REPO, trust_remote_code=True)
model = AutoModelForSequenceClassification.from_pretrained(REPO, trust_remote_code=True).eval()

# Input format used in training: "claim | evidence" (drop the pipe if one side is empty)
text = "Một tuyên bố cần kiểm chứng | bằng chứng liên quan"
enc = tok(text, truncation=True, max_length=256, return_tensors="pt")
with torch.no_grad():
    logits = model(**enc).logits
pred = int(logits.argmax(-1))
print(model.config.id2label[pred])   # 'fake' or 'real'

Note: training used head-tail truncation (ids[:127] + ids[-127:]) at max_length=256. Plain truncation above is fine for most short inputs.

Citation

Distilled within an undergraduate thesis on cross-lingual fake-news detection (EN→VI). Teacher: XLM-RoBERTa-Large (vi_test F1 = 0.8858).