language:
- ar
- en
tags:
- token-classification
- ner
- pii
- arabic
- arabic-dialect
- algeria
- privacy
license: apache-2.0
base_model: aubmindlab/bert-base-arabertv02
pipeline_tag: token-classification
Arabic PII Detector
Fine-tuned token classification model that detects Personally Identifiable Information (PII) in Arabic, English, and mixed Arabic/English text. Covers Modern Standard Arabic, Gulf, Egyptian, and Algerian dialects.
Detected Entity Types
| Tag | Description |
|---|---|
PERSON |
Full name / person name |
EMAIL |
Email address |
PHONE_NUMBER |
Phone number |
ADDRESS |
Physical address |
ACCOUNT_NUMBER |
Account number (internal / short) |
BANK_ACCOUNT_NUMBER |
Bank account number (longer numeric) |
IBAN |
International Bank Account Number |
Base Model
aubmindlab/bert-base-arabertv02 โ selected after evaluating multiple alternatives:
- ~135M parameters (12 transformer layers, hidden size 768)
- Pretrained on large Arabic corpora covering MSA and dialectal Arabic
- Handles code-switching with English natively
- CPU ONNX INT8 expected p95 ~60โ90ms , well within the <150ms latency budget
- Has safetensors format โ
Why not bert-large? aubmindlab/bert-large-arabertv02 (340M params) was tested first , but failed the CPU latency target (p95=239ms with ONNX INT8 on AMD Ryzen 5 3500U).
Why not mdeberta-v3-base? DeBERTa's disentangled attention adds CPU overhead , estimated p95 150โ250ms, borderline at best. Not worth the complexity.
Training Data
Synthetic training data generated by scripts/prepare_data.py:
- ~130 sentence templates covering single entity, multi entity, code-switched, and Algerian dialect cases
- Diverse entity value pools (
scripts/entity_pools.py):- Names: Arabic (MSA + Gulf + Egyptian + Algerian), Berber-influenced.
- Phones: Egypt, Saudi, UAE, Algeria (+213 / 05xx/06xx/07xx)
- IBANs: valid mod-97 checksums for EG/SA/AE/JO/KW/QA/BH/DZ
- Emails: includes
.dz,.fr,yahoo.fr,hotmail.fr,univ-alger.dz - Addresses: 20 Algerian cities, Algerian districts and street names, plus Egypt/Saudi/UAE
- 25% negative examples : hard distractors: tax IDs (NIF), SWIFT codes, IMEI numbers, decree numbers, invoice refs, customs codes
Evaluation split strategy: All three splits (train/val/test) use structurally distinct held-out templates. Val and test templates are of equal difficulty ; a mix of simple, ambiguous, and hard multi-entity cases (up to 4 entities), including the ACCOUNT_NUMBER vs BANK_ACCOUNT_NUMBER confusion pair in both pools. This ensures val F1 during training is honest and comparable to test F1.
| Split | Examples | Templates |
|---|---|---|
| train | 50,000 | ~110 (from TEMPLATES pool) |
| validation | 2,000 | 21 (VAL_ONLY_TEMPLATES, held-out) |
| test | 2,000 | 21 (TEST_ONLY_TEMPLATES, held-out) |
Label Schema
BIO tagging scheme with 15 labels:
O, B-PERSON, I-PERSON, B-EMAIL, I-EMAIL,
B-PHONE_NUMBER, I-PHONE_NUMBER, B-ADDRESS, I-ADDRESS,
B-ACCOUNT_NUMBER, I-ACCOUNT_NUMBER,
B-BANK_ACCOUNT_NUMBER, I-BANK_ACCOUNT_NUMBER,
B-IBAN, I-IBAN
Training Configuration
Training hardware: NVIDIA Tesla T4 GPU (Google Colab)
| Hyperparameter | Value | Reasoning |
|---|---|---|
| Base model | aubmindlab/bert-base-arabertv02 | |
| Epochs | 12 (early stopping) | Ceiling ,early stopping finds the peak |
| Early stopping patience | 3 | Cosine schedule can dip before recovering |
| Batch size | 32 | bert-base fits larger batches; more stable gradients |
| Learning rate | 3e-5 | bert-base needs higher lr than bert-large |
| LR scheduler | Cosine | Stays near peak lr longer, then decays smoothly โ outperforms linear on NER by 0.5โ1.5 F1 points |
| Warmup ratio | 0.06 | Short warmup; over-warming wastes steps with 50K examples |
| Weight decay | 0.01 | Standard for bert-base; 0.1 (used for bert-large) caused underfit |
| Label smoothing | 0.0 | bert-base is underconfident; smoothing hurt recall |
| Max sequence length | 256 | |
| Optimizer | AdamW | |
| fp16 | true (GPU training) | |
| Seed | 42 |
Key insight: hyperparameters differ deliberately from bert-large. bert-large (340M params) needed heavy regularization (weight_decay=0.1, dropout=0.2, label_smoothing=0.1) to prevent overfitting synthetic data. Applying those same settings to bert-base (135M params) caused underfit , the smaller model needs lighter regularization and a higher learning rate to reach its capacity ceiling.
Preprocessing
- Arabic-Indic digits (
ู ูกูขูฃ...) normalized to Western digits (0123...) before tokenization 1:1 character mapping preserves original offsets - Fast (Rust-based) tokenizer for speed
Postprocessing
- BIO decoding to spans
- IBAN mod-97 checksum and email format checks adjust confidence (never override model prediction)
- Trailing whitespace/punctuation trimmed from entity spans
Performance
Training & benchmark hardware: NVIDIA Tesla T4 GPU (Google Colab)
Evaluated on 2000 examples using held-out templates never seen during training:
| Class | Precision | Recall | F1 | Support |
|---|---|---|---|---|
| 1.000 | 1.000 | 1.000 | 283 | |
| PERSON | 0.996 | 0.996 | 0.996 | 516 |
| IBAN | 0.991 | 1.000 | 0.995 | 317 |
| PHONE_NUMBER | 0.994 | 0.991 | 0.993 | 350 |
| BANK_ACCOUNT_NUMBER | 0.968 | 1.000 | 0.984 | 273 |
| ACCOUNT_NUMBER | 0.990 | 0.958 | 0.974 | 214 |
| ADDRESS | 0.990 | 0.990 | 0.990 | 296 |
| Overall | 0.9916 |
| Metric | Before fine-tuning | PyTorch (T4 GPU) | ONNX INT8 (Ryzen 5 CPU) |
|---|---|---|---|
| Overall F1 | 0.0006 | 0.9916 | 0.9916 |
| Val F1 | โ | 0.9682 | โ |
| Precision | โ | 0.9907 | 0.9907 |
| Recall | โ | 0.9924 | 0.9924 |
| Latency p50 | โ | 7.35ms | 57.29ms |
| Latency p95 | โ | 7.97ms | 96.38ms |
| Within 150ms | โ | โ | โ |
Hardware notes:
- Training + GPU benchmark: NVIDIA Tesla T4 (Google Colab)
- CPU benchmark: ThinkPad AMD Ryzen 5 3500U, 4 cores, no discrete GPU โ typical CPU-only deployment machine
- ONNX quantization uses AVX2 config (matches Ryzen 5 3500U instruction set , not AVX-512 which would silently fall back to a slower path)
On the val/test F1 gap: Val F1 (0.9682) and test F1 (0.9916) both use structurally distinct held-out templates of equal difficulty. The 2.3-point gap reflects natural random variation in generated entity values across the two pools โ not data leakage. In real-world deployment expect performance closer to val F1 (0.9682) since real text contains noise not present in synthetic data.
Intended Use
- Redaction pipelines for Arabic customer support data, chat logs, documents
- Pre-processing before sending text to LLMs
- Privacy compliance workflows in North Africa (Algeria, Morocco, Tunisia) and the broader Arab world
Limitations
- Synthetic data domain gap: real text has OCR errors, dialectal variations, and structural patterns not covered by templates
- ACCOUNT_NUMBER vs BANK_ACCOUNT_NUMBER: distinguished by length and context โ borderline cases may be confused
- No PII types beyond the 7 listed: national IDs, passport numbers, credit cards are NOT detected
- Maximum input length: 256 tokens
How to Use
from transformers import AutoTokenizer, AutoModelForTokenClassification
tokenizer = AutoTokenizer.from_pretrained("C-Ilyas/arabic-pii-detector")
model = AutoModelForTokenClassification.from_pretrained("C-Ilyas/arabic-pii-detector")
# See scripts/infer.py for full pipeline with BIO decoding and redaction
# CLI
python scripts/infer.py --text "ุงุณู
ู ู
ุญู
ุฏ ุฃุญู
ุฏ ูุฑูู
ุชูููููู 01012345678" --pretty
python scripts/infer.py --text "ุฑุงูู ุณููุงู ุจูุฒูุฏ ูุฑูู
ู 0698123456 ู
ู ุงูุฌุฒุงุฆุฑ ุงูุนุงุตู
ุฉ" --pretty
Evaluation
See results/after_finetuning_metrics.json for full per-class metrics, results/latency_report.json for latency profiling, and results_onnx/ for CPU ONNX benchmark results.