phuy1125/xlmr-large-fakenews-teacher

XLM-RoBERTa-Large — Cross-lingual Fake-News Detection (Teacher)

Teacher model for cross-lingual (EN → VI) fake-news / claim verification. Trained on English first (M1), then adapted to Vietnamese with a mixed LoRA stage (Stage-3, r=8, α=16); the LoRA delta was merged back into the backbone. This is the best seed (42) teacher used to distill the student models.

• Backbone: xlm-roberta-large (~560M params total)
• Labels: 0 = fake (tin giả), 1 = real (tin thật)

Results (best seed = 42)

Split	F1-macro
VI test	0.8858
EN test	0.9209
VI val	0.8617

Distilled students

• phuy1125/xlmr-base-fakenews-kd (F1 0.8401)
• phuy1125/mminilm-l12-fakenews-kd (F1 0.8450)
• phuy1125/mminilm-l6-fakenews-kd (F1 0.8262)
• phuy1125/xlmr-large-fakenews-lora (Stage-3 LoRA adapter)

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

REPO = "phuy1125/xlmr-large-fakenews-teacher"
tok = AutoTokenizer.from_pretrained(REPO, trust_remote_code=True)
model = AutoModelForSequenceClassification.from_pretrained(REPO, trust_remote_code=True).eval()

# Input format used in training: "claim | evidence"
text = "Một tuyên bố cần kiểm chứng | bằng chứng liên quan"
enc = tok(text, truncation=True, max_length=256, return_tensors="pt")
with torch.no_grad():
    logits = model(**enc).logits
print(model.config.id2label[int(logits.argmax(-1))])   # 'fake' or 'real'

Note: training used head-tail truncation (ids[:127] + ids[-127:]) at max_length=256.