README.md

---
license: mit
base_model: FacebookAI/xlm-roberta-base
library_name: transformers
pipeline_tag: text-classification
language:
  - en
  - zh
  - vi
tags:
  - text-classification
  - ai-text-detection
  - ai-content-forensics
  - xlm-roberta
  - multilingual
metrics:
  - f1
model-index:
  - name: multilingual-ai-human-detector_xlm-roberta-base
    results:
      - task:
          type: text-classification
          name: AI-generated vs human-written text detection
        dataset:
          name: Multilingual QA corpus (HC3 / HC3-Chinese / Vietnamese Reddit)
          type: custom
        metrics:
          - type: f1
            value: 0.9710
            name: F1 (overall, EN+ZH+VI test set)
          - type: f1
            value: 0.9890
            name: F1 (English)
          - type: f1
            value: 0.9462
            name: F1 (Chinese)
          - type: f1
            value: 0.9783
            name: F1 (Vietnamese)
---

# Multilingual AI-vs-Human Text Detector — XLM-RoBERTa-base

A binary classifier that detects whether a passage was **written by a human or generated by an AI model**, fine-tuned from **XLM-RoBERTa-base** to work across **English, Chinese, and Vietnamese** with a single checkpoint.

Built as an **AI content forensics** research project: the full system — data pipeline, statistical baselines, per-language evaluation, REST API, and web demo — lives in the companion repository.

- 📦 **GitHub (full project):** https://github.com/vutuongvy101/multilingual-ai-human-text-detection
- 👩‍💻 **Author:** Tuong Vy Vu — [GitHub](https://github.com/vutuongvy101) · [LinkedIn](https://www.linkedin.com/in/vy-vu-260153177)

## Why a multilingual detector?

Monolingual detectors fail badly outside their pre-training language: in the same experimental setup, an English-only DistilBERT drops to **F1 0.784 on Chinese** (vs 0.977 on English) because Chinese characters tokenise into meaningless subword fragments. This model uses multilingual pre-training to hold **F1 0.946–0.989 across all three languages** — evidence that cross-lingual AI-content detection requires multilingual representations, not per-language models.

## Evaluation

Test-set F1 (70/15/15 prompt-level split, seed 42; 90 test samples per language, 270 total):

| Language | F1 |
|---|---|
| English | **0.9890** |
| Vietnamese | 0.9783 |
| Chinese | 0.9462 |
| **Overall** | **0.9710** |

Comparison against baselines trained on the same data (full table, confusion matrices, and analysis in the [GitHub README](https://github.com/vutuongvy101/multilingual-ai-human-text-detection#results)):

| Model | Overall F1 |
|---|---|
| Logistic Regression (TF-IDF) | 0.9776 |
| **XLM-RoBERTa (this model)** | **0.9710** |
| Multinomial NB (TF-IDF) | 0.9181 |
| DistilBERT | 0.8960 |

Note that the TF-IDF logistic regression baseline is competitive in-domain — the value of this transformer model is expected in robustness to paraphrase and vocabulary shift, which n-gram features cannot capture (see Limitations).

## Training

- **Base model:** `FacebookAI/xlm-roberta-base` (~279M params), sequence classification head, F32.
- **Data:** 900 QA pairs (300 per language), each with one human and one AI answer:
  - English — HC3 (`reddit_eli5`)
  - Chinese — HC3-Chinese (`open_qa`)
  - Vietnamese — crawled from Vietnamese Reddit communities
  - AI answers generated by **Qwen2.5-1.5B-Instruct**
- **Setup:** fine-tuned 3 epochs, learning rate `2e-5`, `max_length=256`, prompt-level 70/15/15 train/val/test split (seed 42) so no question appears in both train and test.
- **Reproduce:** `python scripts/train_transformer.py --model-name xlm-roberta-base` in the GitHub repo.

## Usage

```python
from transformers import pipeline

detector = pipeline(
    "text-classification",
    model="bibbbu/multilingual-ai-human-detector_xlm-roberta-base",
)

texts = [
    "Honestly I just left it overnight and it worked fine, no idea why lol",
    "There are several important factors to consider when addressing this question.",
]
print(detector(texts))
# [{'label': 'human', 'score': ...}, {'label': 'ai', 'score': ...}]
```

For batch inference, a FastAPI server, and a Streamlit demo, see the [GitHub repository](https://github.com/vutuongvy101/multilingual-ai-human-text-detection).

## Intended use & limitations

**Intended use:** research on cross-lingual AI-text detection; educational and portfolio use; experimentation with content-authenticity pipelines.

**Limitations — read before relying on predictions:**

- ⚠️ **Do not use this model alone for high-stakes decisions** (academic-misconduct accusations, content moderation enforcement, hiring). AI-text detectors produce false positives, and non-native writers are a known false-positive risk for detectors in general.
- **Generator specificity:** AI-labelled training text comes from a single generator (Qwen2.5-1.5B-Instruct). Detection of text from other models (GPT-4-class, Claude, Gemini) is untested and likely weaker.
- **Domain shift:** trained on QA-style forum answers; performance on news, legal, academic, or social-media text is untested.
- **Small evaluation set:** 270 test samples total; strong scores should be confirmed on larger, held-out domains before any production use.
- **Language coverage:** English, Chinese, Vietnamese only.

## Citation

```bibtex
@misc{vu2024multilingual,
  title  = {Multilingual AI-Human Text Detection},
  author = {Vu, Tuong Vy},
  year   = {2024},
  url    = {https://github.com/vutuongvy101/multilingual-ai-human-text-detection}
}
```