---
library_name: transformers
license: apache-2.0

pipeline_tag: text-classification

language:
- ar
- en
- fa
- he
- ru
- bn
- zh
- ko
- tr
- sw
- am
- so
- id
- ms
- es
- fr
- de
- pt
- it
- ur
- da

tags:
- hate-speech-detection
- multilingual
- text-classification
- mbert
- cross-lingual
- zero-shot-learning
- low-resource-languages
- transformers

metrics:
- accuracy
- f1

base_model:
- jhu-clsp/mmBERT-base 
---

# mmBERT: Advanced Multilingual Encoder Model

## Model Description

This model is a multilingual hate speech detection model fine-tuned from **jhu-clsp/mmBERT-base (mmBERT)** using datasets from 21 languages belonging to multiple language families and writing systems. The model aims to support robust multilingual hate speech classification and cross-lingual generalization across both high-resource and low-resource languages.

The model performs binary classification:

- Hate Speech
- Non-Hate Speech

## Model Details

- **Developed by:** Ghadeer Albadani
- **Base Model:** bert-base-multilingual-cased (mBERT)
- **Model Type:** Transformer-based Text Classification
- **Task:** Multilingual Hate Speech Detection
- **Framework:** Hugging Face Transformers
- **Training Languages:** 21 Languages
- **License:** Apache-2.0

## Supported Languages

The model was fine-tuned using the following languages:

- Arabic
- Hebrew
- Persian
- English
- French
- German
- Spanish
- Portuguese
- Italian
- Danish
- Russian
- Turkish
- Bengali
- Chinese
- Korean
- Malay
- Indonesian
- Swahili
- Amharic
- Somali
- Roman Urdu

These languages represent diverse language families and writing systems, enabling multilingual hate speech detection and cross-lingual generalization.

## Intended Use

### Direct Use

The model can be used for:

- Multilingual hate speech detection
- Toxic content classification
- Social media moderation
- Multilingual NLP research
- Cross-lingual text classification

### Downstream Applications

- Content moderation systems
- Hate speech monitoring platforms
- Social media analytics
- Cross-lingual NLP applications
- Low-resource language research

### Out-of-Scope Use

This model should not be used as:

- A legal decision-making system
- A replacement for human moderation
- A profiling tool for individuals or groups
- A fully automated moderation system without human oversight

## Benchmark Performance

### Evaluation Results

The model was evaluated independently on multilingual hate speech datasets covering 20 languages.

| Language | Accuracy | F1-Score | Notes |
|-----------|----------|----------|--------|
| Arabic | 0.82 | 0.82 | Moderate performance |
| Persian | 0.94 | 0.94 | Excellent performance |
| Hebrew | 0.76 | 0.76 | Lower performance |
| Bengali | 0.89 | 0.89 | Good performance |
| Korean | 0.76 | 0.76 | Lower performance |
| Chinese | 0.81 | 0.81 | Moderate performance |
| Russian | 0.89 | 0.89 | Good performance |
| Spanish | 0.82 | 0.82 | Moderate performance |
| Indonesian | 0.94 | 0.94 | Excellent performance |
| Turkish | 0.85 | 0.85 | Good performance |
| English | 0.89 | 0.89 | Good performance |
| French | 0.85 | 0.85 | Good performance |
| German | 0.64 | 0.63 | Poor performance |
| Portuguese | 0.71 | 0.71 | Moderate–poor performance |
| Malay | 0.65 | 0.65 | Poor performance |
| Italian | 0.80 | 0.79 | Moderate performance |
| Roman Urdu | 0.82 | 0.82 | Moderate performance |
| Amharic | 0.77 | 0.76 | Moderate performance |
| Swahili | 0.90 | 0.90 | Very good performance |
| Somali | 0.71 | 0.71 | Moderate–poor performance |

### Summary

The model demonstrated strong multilingual hate speech detection capabilities across a diverse set of languages. The highest performance was achieved on Persian and Indonesian, both obtaining an Accuracy and F1-score of 0.94, followed by Swahili (0.90), English (0.89), Russian (0.89), and Bengali (0.89). These results indicate that the model successfully learned language-independent hate speech representations despite substantial linguistic diversity among the training languages.

## Training Data

The model was fine-tuned on multilingual hate speech datasets collected from multiple publicly available sources covering 21 languages.

The datasets contain two labels:

- Hate Speech
- Non-Hate Speech

Data preprocessing included cleaning, normalization, tokenization, and label standardization.

## Training Procedure

### Hyperparameters

| Parameter | Value |
|------------|---------|
| Epochs | 2 |
| Learning Rate | 2e-5 |
| Batch Size | 32 |
| Optimizer Epsilon | 1e-8 |
| Maximum Sequence Length | 512 |

### Hardware

Training was performed using:

- GPU: NVIDIA L4
- GPU Memory: 24 GB

### Software

- Python
- PyTorch
- Hugging Face Transformers
- CUDA

## Evaluation Metrics

The model was evaluated using:

- Accuracy
- Precision
- Recall
- F1-Score
- Macro F1

## Bias, Risks, and Limitations

Although the model was trained on multilingual datasets from diverse language families, performance may vary depending on:

- Dataset quality
- Annotation consistency
- Cultural interpretation of hate speech
- Domain differences
- Language-specific characteristics

Users should evaluate the model carefully before deployment in real-world moderation systems.

## How to Use

```python
from transformers import AutoTokenizer
from transformers import AutoModelForSequenceClassification
from transformers import pipeline

model_name = "GhadeerALbadani/mmbert-Multilingual_detection_of_hate_speech"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

classifier = pipeline(
    "text-classification",
    model=model,
    tokenizer=tokenizer
)

text = "I hate all people from this group."

result = classifier(text)

print(result)
```

## Model Architecture

The model is based on **(mBERT)**.

Architecture details:

-Transformer Encoder Architecture
-22 Transformer Layers
-Hidden Size: 768
-Intermediate Size: 1152
-12 Attention Heads
-Approximately 307 Million Parameters
-110 Million Non-embedding Parameters
-Maximum Sequence Length: 8192 Tokens
-Vocabulary Size: 256,000 Tokens
-Gemma 2 Tokenizer
-Pretrained on multilingual web data, Wikipedia, academic papers, code repositories, and community discussions
-Supports multilingual understanding and cross-lingual transfer learning
## Research Context

This model was developed as part of research on multilingual hate speech detection, cross-lingual transfer learning, and multilingual natural language processing.

The research investigates:

- Multilingual training
- Cross-lingual transfer
- Zero-shot learning
- Hate speech detection in low-resource languages
- Language-independent representation learning

## Citation

If you use this model in your research, please cite:

```bibtex
@misc{albadani2026mmbert,
  author = {Ghadeer Albadani},
  title = {mmBERT: Multilingual Detection of Hate Speech},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/GhadeerALbadani/mmbert-Multilingual_detection_of_hate_speech}
}
```

## Contact

**Author:** Ghadeer Albadani

**Model Repository:**
https://huggingface.co/GhadeerALbadani/mmbert-Multilingual_detection_of_hate_speech