C-Ilyas/arabic-pii-detector

Arabic PII Detector

Fine-tuned token classification model that detects Personally Identifiable Information (PII) in Arabic, English, and mixed Arabic/English text. Covers Modern Standard Arabic, Gulf, Egyptian, and Algerian dialects.

Detected Entity Types

Tag	Description
PERSON	Full name / person name
EMAIL	Email address
PHONE_NUMBER	Phone number
ADDRESS	Physical address
ACCOUNT_NUMBER	Account number (internal / short)
BANK_ACCOUNT_NUMBER	Bank account number (longer numeric)
IBAN	International Bank Account Number

Base Model

aubmindlab/bert-base-arabertv02 — selected after evaluating multiple alternatives:

• ~135M parameters (12 transformer layers, hidden size 768)
• Pretrained on large Arabic corpora covering MSA and dialectal Arabic
• Handles code-switching with English natively
• CPU ONNX INT8 expected p95 ~60–90ms , well within the <150ms latency budget
• Has safetensors format ✓

Why not bert-large? aubmindlab/bert-large-arabertv02 (340M params) was tested first , but failed the CPU latency target (p95=239ms with ONNX INT8 on AMD Ryzen 5 3500U).

Why not mdeberta-v3-base? DeBERTa's disentangled attention adds CPU overhead , estimated p95 150–250ms, borderline at best. Not worth the complexity.

Training Data

Synthetic training data generated by scripts/prepare_data.py:

• ~130 sentence templates covering single entity, multi entity, code-switched, and Algerian dialect cases
• Diverse entity value pools (scripts/entity_pools.py):
  • Names: Arabic (MSA + Gulf + Egyptian + Algerian), Berber-influenced.
  • Phones: Egypt, Saudi, UAE, Algeria (+213 / 05xx/06xx/07xx)
  • IBANs: valid mod-97 checksums for EG/SA/AE/JO/KW/QA/BH/DZ
  • Emails: includes .dz, .fr, yahoo.fr, hotmail.fr, univ-alger.dz
  • Addresses: 20 Algerian cities, Algerian districts and street names, plus Egypt/Saudi/UAE
• 25% negative examples : hard distractors: tax IDs (NIF), SWIFT codes, IMEI numbers, decree numbers, invoice refs, customs codes

Evaluation split strategy: All three splits (train/val/test) use structurally distinct held-out templates. Val and test templates are of equal difficulty ; a mix of simple, ambiguous, and hard multi-entity cases (up to 4 entities), including the ACCOUNT_NUMBER vs BANK_ACCOUNT_NUMBER confusion pair in both pools. This ensures val F1 during training is honest and comparable to test F1.

Split	Examples	Templates
train	50,000	~110 (from TEMPLATES pool)
validation	2,000	21 (VAL_ONLY_TEMPLATES, held-out)
test	2,000	21 (TEST_ONLY_TEMPLATES, held-out)

Label Schema

BIO tagging scheme with 15 labels:

O, B-PERSON, I-PERSON, B-EMAIL, I-EMAIL,
B-PHONE_NUMBER, I-PHONE_NUMBER, B-ADDRESS, I-ADDRESS,
B-ACCOUNT_NUMBER, I-ACCOUNT_NUMBER,
B-BANK_ACCOUNT_NUMBER, I-BANK_ACCOUNT_NUMBER,
B-IBAN, I-IBAN

Training Configuration

Training hardware: NVIDIA Tesla T4 GPU (Google Colab)

Hyperparameter	Value	Reasoning
Base model	aubmindlab/bert-base-arabertv02
Epochs	12 (early stopping)	Ceiling ,early stopping finds the peak
Early stopping patience	3	Cosine schedule can dip before recovering
Batch size	32	bert-base fits larger batches; more stable gradients
Learning rate	3e-5	bert-base needs higher lr than bert-large
LR scheduler	Cosine	Stays near peak lr longer, then decays smoothly — outperforms linear on NER by 0.5–1.5 F1 points
Warmup ratio	0.06	Short warmup; over-warming wastes steps with 50K examples
Weight decay	0.01	Standard for bert-base; 0.1 (used for bert-large) caused underfit
Label smoothing	0.0	bert-base is underconfident; smoothing hurt recall
Max sequence length	256
Optimizer	AdamW
fp16	true (GPU training)
Seed	42

Key insight: hyperparameters differ deliberately from bert-large. bert-large (340M params) needed heavy regularization (weight_decay=0.1, dropout=0.2, label_smoothing=0.1) to prevent overfitting synthetic data. Applying those same settings to bert-base (135M params) caused underfit , the smaller model needs lighter regularization and a higher learning rate to reach its capacity ceiling.

Preprocessing

• Arabic-Indic digits (٠١٢٣...) normalized to Western digits (0123...) before tokenization 1:1 character mapping preserves original offsets
• Fast (Rust-based) tokenizer for speed

Postprocessing

• BIO decoding to spans
• IBAN mod-97 checksum and email format checks adjust confidence (never override model prediction)
• Trailing whitespace/punctuation trimmed from entity spans

Performance

Training & benchmark hardware: NVIDIA Tesla T4 GPU (Google Colab)

Evaluated on 2000 examples using held-out templates never seen during training:

Class	Precision	Recall	F1	Support
EMAIL	1.000	1.000	1.000	283
PERSON	0.996	0.996	0.996	516
IBAN	0.991	1.000	0.995	317
PHONE_NUMBER	0.994	0.991	0.993	350
BANK_ACCOUNT_NUMBER	0.968	1.000	0.984	273
ACCOUNT_NUMBER	0.990	0.958	0.974	214
ADDRESS	0.990	0.990	0.990	296
Overall			0.9916

Metric	Before fine-tuning	PyTorch (T4 GPU)	ONNX INT8 (Ryzen 5 CPU)
Overall F1	0.0006	0.9916	0.9916
Val F1	—	0.9682	—
Precision	—	0.9907	0.9907
Recall	—	0.9924	0.9924
Latency p50	—	7.35ms	57.29ms
Latency p95	—	7.97ms	96.38ms
Within 150ms	—	✓	✓

Hardware notes:

• Training + GPU benchmark: NVIDIA Tesla T4 (Google Colab)
• CPU benchmark: ThinkPad AMD Ryzen 5 3500U, 4 cores, no discrete GPU — typical CPU-only deployment machine
• ONNX quantization uses AVX2 config (matches Ryzen 5 3500U instruction set , not AVX-512 which would silently fall back to a slower path)

On the val/test F1 gap: Val F1 (0.9682) and test F1 (0.9916) both use structurally distinct held-out templates of equal difficulty. The 2.3-point gap reflects natural random variation in generated entity values across the two pools — not data leakage. In real-world deployment expect performance closer to val F1 (0.9682) since real text contains noise not present in synthetic data.

Intended Use

• Redaction pipelines for Arabic customer support data, chat logs, documents
• Pre-processing before sending text to LLMs
• Privacy compliance workflows in North Africa (Algeria, Morocco, Tunisia) and the broader Arab world

Limitations

• Synthetic data domain gap: real text has OCR errors, dialectal variations, and structural patterns not covered by templates
• ACCOUNT_NUMBER vs BANK_ACCOUNT_NUMBER: distinguished by length and context — borderline cases may be confused
• No PII types beyond the 7 listed: national IDs, passport numbers, credit cards are NOT detected
• Maximum input length: 256 tokens

How to Use

from transformers import AutoTokenizer, AutoModelForTokenClassification

tokenizer = AutoTokenizer.from_pretrained("C-Ilyas/arabic-pii-detector")
model = AutoModelForTokenClassification.from_pretrained("C-Ilyas/arabic-pii-detector")
# See scripts/infer.py for full pipeline with BIO decoding and redaction

# CLI
python scripts/infer.py --text "اسمي محمد أحمد ورقم تليفوني 01012345678" --pretty
python scripts/infer.py --text "راني سفيان بوزيد ورقمي 0698123456 من الجزائر العاصمة" --pretty

Evaluation

See results/after_finetuning_metrics.json for full per-class metrics, results/latency_report.json for latency profiling, and results_onnx/ for CPU ONNX benchmark results.