README.md

---
language:
  - ar
  - en
tags:
  - token-classification
  - ner
  - pii
  - arabic
  - arabic-dialect
  - algeria
  - privacy
license: apache-2.0
base_model: aubmindlab/bert-base-arabertv02
pipeline_tag: token-classification
---
# Arabic PII Detector

Fine-tuned token classification model that detects Personally Identifiable Information (PII) in Arabic, English, and mixed Arabic/English text. Covers Modern Standard Arabic, Gulf, Egyptian, and Algerian dialects.

## Detected Entity Types

| Tag                     | Description                          |
| ----------------------- | ------------------------------------ |
| `PERSON`              | Full name / person name              |
| `EMAIL`               | Email address                        |
| `PHONE_NUMBER`        | Phone number                         |
| `ADDRESS`             | Physical address                     |
| `ACCOUNT_NUMBER`      | Account number (internal / short)    |
| `BANK_ACCOUNT_NUMBER` | Bank account number (longer numeric) |
| `IBAN`                | International Bank Account Number    |

## Base Model

**aubmindlab/bert-base-arabertv02** — selected after evaluating multiple alternatives:

- ~135M parameters (12 transformer layers, hidden size 768)
- Pretrained on large Arabic corpora covering MSA and dialectal Arabic
- Handles code-switching with English natively
- CPU ONNX INT8 expected p95 ~60–90ms , well within the <150ms latency budget
- Has safetensors format ✓

**Why not bert-large?** `aubmindlab/bert-large-arabertv02` (340M params) was tested first , but failed the CPU latency target (p95=239ms with ONNX INT8 on AMD Ryzen 5 3500U).

**Why not mdeberta-v3-base?** DeBERTa's disentangled attention adds CPU overhead , estimated p95 150–250ms, borderline at best. Not worth the complexity.

## Training Data

**Synthetic** training data generated by `scripts/prepare_data.py`:

- **~130 sentence templates** covering single entity, multi entity, code-switched, and Algerian dialect cases
- **Diverse entity value pools** (`scripts/entity_pools.py`):
  - Names: Arabic (MSA + Gulf + Egyptian + Algerian), Berber-influenced.
  - Phones: Egypt, Saudi, UAE, **Algeria (+213 / 05xx/06xx/07xx)**
  - IBANs: valid mod-97 checksums for EG/SA/AE/JO/KW/QA/BH/**DZ**
  - Emails: includes `.dz`, `.fr`, `yahoo.fr`, `hotmail.fr`, `univ-alger.dz`
  - Addresses: 20 Algerian cities, Algerian districts and street names, plus Egypt/Saudi/UAE
- **25% negative examples** : hard distractors: tax IDs (NIF), SWIFT codes, IMEI numbers, decree numbers, invoice refs, customs codes

**Evaluation split strategy**: All three splits (train/val/test) use structurally distinct held-out templates. Val and test templates are of equal difficulty ; a mix of simple, ambiguous, and hard multi-entity cases (up to 4 entities), including the `ACCOUNT_NUMBER` vs `BANK_ACCOUNT_NUMBER` confusion pair in both pools. This ensures val F1 during training is honest and comparable to test F1.

| Split      | Examples | Templates                          |
| ---------- | -------- | ---------------------------------- |
| train      | 50,000   | ~110 (from TEMPLATES pool)         |
| validation | 2,000    | 21 (VAL_ONLY_TEMPLATES, held-out)  |
| test       | 2,000    | 21 (TEST_ONLY_TEMPLATES, held-out) |

## Label Schema

BIO tagging scheme with 15 labels:

```
O, B-PERSON, I-PERSON, B-EMAIL, I-EMAIL,
B-PHONE_NUMBER, I-PHONE_NUMBER, B-ADDRESS, I-ADDRESS,
B-ACCOUNT_NUMBER, I-ACCOUNT_NUMBER,
B-BANK_ACCOUNT_NUMBER, I-BANK_ACCOUNT_NUMBER,
B-IBAN, I-IBAN
```

## Training Configuration

**Training hardware**: NVIDIA Tesla T4 GPU (Google Colab)

| Hyperparameter          | Value                           | Reasoning                                                                                          |
| ----------------------- | ------------------------------- | -------------------------------------------------------------------------------------------------- |
| Base model              | aubmindlab/bert-base-arabertv02 |                                                                                                    |
| Epochs                  | 12 (early stopping)             | Ceiling ,early stopping finds the peak                                                           |
| Early stopping patience | 3                               | Cosine schedule can dip before recovering                                                          |
| Batch size              | 32                              | bert-base fits larger batches; more stable gradients                                               |
| Learning rate           | 3e-5                            | bert-base needs higher lr than bert-large                                                          |
| LR scheduler            | **Cosine**                | Stays near peak lr longer, then decays smoothly — outperforms linear on NER by 0.5–1.5 F1 points |
| Warmup ratio            | 0.06                            | Short warmup; over-warming wastes steps with 50K examples                                          |
| Weight decay            | 0.01                            | Standard for bert-base; 0.1 (used for bert-large) caused underfit                                  |
| Label smoothing         | 0.0                             | bert-base is underconfident; smoothing hurt recall                                                 |
| Max sequence length     | 256                             |                                                                                                    |
| Optimizer               | AdamW                           |                                                                                                    |
| fp16                    | true (GPU training)             |                                                                                                    |
| Seed                    | 42                              |                                                                                                    |

**Key insight**: hyperparameters differ deliberately from bert-large. bert-large (340M params) needed heavy regularization (weight_decay=0.1, dropout=0.2, label_smoothing=0.1) to prevent overfitting synthetic data. Applying those same settings to bert-base (135M params) caused underfit , the smaller model needs lighter regularization and a higher learning rate to reach its capacity ceiling.

## Preprocessing

- Arabic-Indic digits (`٠١٢٣...`) normalized to Western digits (`0123...`) before tokenization  1:1 character mapping preserves original offsets
- Fast (Rust-based) tokenizer for speed

## Postprocessing

- BIO decoding to spans
- IBAN mod-97 checksum and email format checks adjust confidence (never override model prediction)
- Trailing whitespace/punctuation trimmed from entity spans

## Performance

**Training & benchmark hardware**: NVIDIA Tesla T4 GPU (Google Colab)

Evaluated on 2000 examples using **held-out templates never seen during training**:

| Class               | Precision | Recall | F1               | Support |
| ------------------- | --------- | ------ | ---------------- | ------- |
| EMAIL               | 1.000     | 1.000  | 1.000            | 283     |
| PERSON              | 0.996     | 0.996  | 0.996            | 516     |
| IBAN                | 0.991     | 1.000  | 0.995            | 317     |
| PHONE_NUMBER        | 0.994     | 0.991  | 0.993            | 350     |
| BANK_ACCOUNT_NUMBER | 0.968     | 1.000  | 0.984            | 273     |
| ACCOUNT_NUMBER      | 0.990     | 0.958  | 0.974            | 214     |
| ADDRESS             | 0.990     | 0.990  | 0.990            | 296     |
| **Overall**   |           |        | **0.9916** |         |

| Metric       | Before fine-tuning | PyTorch (T4 GPU) | ONNX INT8 (Ryzen 5 CPU) |
| ------------ | ------------------ | ---------------- | ----------------------- |
| Overall F1   | 0.0006             | **0.9916** | **0.9916**        |
| Val F1       | —                 | 0.9682           | —                      |
| Precision    | —                 | 0.9907           | 0.9907                  |
| Recall       | —                 | 0.9924           | 0.9924                  |
| Latency p50  | —                 | 7.35ms           | 57.29ms                 |
| Latency p95  | —                 | 7.97ms           | **96.38ms**       |
| Within 150ms | —                 | ✓               | ✓                      |

**Hardware notes**:

- Training + GPU benchmark: NVIDIA Tesla T4 (Google Colab)
- CPU benchmark: ThinkPad AMD Ryzen 5 3500U, 4 cores, no discrete GPU — typical CPU-only deployment machine
- ONNX quantization uses AVX2 config (matches Ryzen 5 3500U instruction set , not AVX-512 which would silently fall back to a slower path)

**On the val/test F1 gap**: Val F1 (0.9682) and test F1 (0.9916) both use structurally distinct held-out templates of equal difficulty. The 2.3-point gap reflects natural random variation in generated entity values across the two pools — not data leakage. In real-world deployment expect performance closer to val F1 (0.9682) since real text contains noise not present in synthetic data.

## Intended Use

- Redaction pipelines for Arabic customer support data, chat logs, documents
- Pre-processing before sending text to LLMs
- Privacy compliance workflows in North Africa (Algeria, Morocco, Tunisia) and the broader Arab world

## Limitations

- **Synthetic data domain gap**: real text has OCR errors, dialectal variations, and structural patterns not covered by templates
- **ACCOUNT_NUMBER vs BANK_ACCOUNT_NUMBER**: distinguished by length and context — borderline cases may be confused
- **No PII types beyond the 7 listed**: national IDs, passport numbers, credit cards are NOT detected
- **Maximum input length**: 256 tokens

## How to Use

```python
from transformers import AutoTokenizer, AutoModelForTokenClassification

tokenizer = AutoTokenizer.from_pretrained("C-Ilyas/arabic-pii-detector")
model = AutoModelForTokenClassification.from_pretrained("C-Ilyas/arabic-pii-detector")
# See scripts/infer.py for full pipeline with BIO decoding and redaction
```

```bash
# CLI
python scripts/infer.py --text "اسمي محمد أحمد ورقم تليفوني 01012345678" --pretty
python scripts/infer.py --text "راني سفيان بوزيد ورقمي 0698123456 من الجزائر العاصمة" --pretty
```

## Evaluation

See `results/after_finetuning_metrics.json` for full per-class metrics, `results/latency_report.json` for latency profiling, and `results_onnx/` for CPU ONNX benchmark results.