README.md

---
language:
- ar
- fr
license: apache-2.0
tags:
- token-classification
- ner
- named-entity-recognition
- algerian-arabic
- darja
- arabizi
- code-switching
- real-estate
- deberta
- deberta-v2
- information-extraction
- low-resource-nlp
pipeline_tag: token-classification
widget:
- text: كاين اف3 اتاج 8 درارية صيغة lsp المهتم يبعثلي في الخاص
  example_title: Arabic Darja listing
- text: >-
    Appartement F4 à vendre Oran centre 120m² 4ème étage acte notarié contact
    [PHONE]
  example_title: French listing
- text: >-
    Salam rani nbi3 fi villa f5 fi hydra alger 350m b jardin w garage te3ha dir
    25 milliards
  example_title: Arabizi listing
datasets:
- 81melody/algerian-realestate-ner-dataset
metrics:
- f1
- precision
- recall
model-index:
- name: algerianDeBERTa-realestate-ner
  results:
  - task:
      type: token-classification
      name: Named Entity Recognition
    dataset:
      name: 81melody/algerian-realestate-ner
      type: 81melody/algerian-realestate-ner
    metrics:
    - type: f1
      value: 0.9672
      name: Test F1 (micro, seqeval)
    - type: precision
      value: 0.9566
      name: Test Precision (micro)
    - type: recall
      value: 0.978
      name: Test Recall (micro)
base_model:
- 81melody/algerianDeBERTa
---

# algerianDeBERTa-realestate-ner

**A Name Entity Recognition model based on AlgerianDeBERTa, finetuned to extract 13 real-estate entities from Algerian Facebook posts**

Handles the exact language mix found on Facebook Marketplace and Algerian classified groups: **Darja** (Algerian dialect), **Arabizi** (Arabic written in Latin script), **French**, and heavy **code-switching**, finetuned on +7k examples of pure algerian real-estate posts from Facebook

---

## Model Highlights

| | |
|---|---|
| **Architecture** | DeBERTa-v2 — 12 layers, hidden=512, 8 heads, 2048 FFN |
| **Base model** | algerianDeBERTa (pre-trained on Algerian web text) |
| **Task** | Token classification — 27 BIO labels, 13 entity types |
| **Languages** | Algerian Darja · Arabizi · French · MSA · Code-switched |
| **Domain** | Real estate classifieds (sales, rentals, land, villas, apartments) |
| **Test F1** | **0.9672** micro (seqeval, strict entity-level) |
| **Best val F1** | **0.9858** |
| **Parameters** | ~60M |
| **License** | Apache 2.0 |

---

## Quick Start

### Option 1 : `pipeline` (standard, recommended for short texts)

Uses `aggregation_strategy="max"`: for each surface word the subword token with the highest entity-class score wins, then consecutive  spans that have the same type are merged automatically

```python
from transformers import pipeline

ner = pipeline(
    "token-classification",
    model="81melody/algerianDeBERTa-realestate-ner",
    aggregation_strategy="max",
)


print(ner("سلام ، خصني اف2 فالعاصمة في ميسوني ولا اودان ولا ديدوش ، في هاد الجويه لي عندو يتوصل معيا في الخاص"))

print(ner("Appartement F4 à vendre Oran centre 120m² 4ème étage acte notarié"))

```

For texts that may exceed 192 tokens, pass sliding-window arguments directly to the pipeline call:

```python
result = ner(
    long_text,
    truncation=True,
    max_length=192,
    stride=64,
)
```

Each result dict contains `entity_group`, `word`, `score`, `start`, `end`.

---

### Option 2 : Manual sliding window inference (production / long texts)

For real estate posts that frequently exceed one chunk + to adapt with The small vocab size of the first version of the base model (30k), the approach below is more robust for production use: it **averages the probability vectors** of overlapping tokens across all chunks, then merges subword pieces that form the same surface word before BIO decoding

This fixes a common artefact where words like `"cherche"` (tokenised as `["cher", "che"]`) or prices like `"1.700"` (tokenised as `["1", ".", "700"]`) get truncated mid-word if a trailing subword happens to predict `O`

```python
from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch
import numpy as np
from typing import List

MODEL_NAME  = "81melody/algerianDeBERTa-realestate-ner"
MAX_SEQ_LEN = 192
STRIDE      = 64

tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
model     = AutoModelForTokenClassification.from_pretrained(MODEL_NAME)
model.eval()
device    = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)


def extract_entities(text: str) -> List[dict]:
   
    enc = tokenizer(
        text,
        return_tensors="pt",
        max_length=MAX_SEQ_LEN,
        stride=STRIDE,
        truncation=True,
        return_overflowing_tokens=True,
        return_offsets_mapping=True,
        padding="max_length",
    )
    enc.pop("overflow_to_sample_mapping", None)
    offsets = enc.pop("offset_mapping")          


    with torch.no_grad():
        logits = model(**{k: v.to(device) for k, v in enc.items()}).logits
    probs     = torch.softmax(logits, dim=-1).cpu().numpy()  
    attention = enc["attention_mask"].numpy()                
    off_np    = offsets.numpy()                             


    char_probs = {}
    for c in range(probs.shape[0]):
        for t in range(off_np.shape[1]):
            if attention[c, t] == 0:
                continue
            cs, ce = int(off_np[c, t, 0]), int(off_np[c, t, 1])
            if cs == 0 and ce == 0:         
                continue
            if cs not in char_probs:
                char_probs[cs] = {"end": ce, "vecs": [probs[c, t]]}
            else:
                char_probs[cs]["vecs"].append(probs[c, t])

    if not char_probs:
        return []

    items = sorted(char_probs.items())

    word_tokens = []
    w_start, w_info = items[0]
    w_end   = w_info["end"]
    w_avg_p = np.mean(w_info["vecs"], axis=0)  

    for cs, info in items[1:]:
        if cs == w_end:                         
            w_end = info["end"]
        else:                                    
            word_tokens.append({"start": w_start, "end": w_end, "avg_p": w_avg_p})
            w_start = cs
            w_end   = info["end"]
            w_avg_p = np.mean(info["vecs"], axis=0)
    word_tokens.append({"start": w_start, "end": w_end, "avg_p": w_avg_p})

    label_map = model.config.id2label
    entities, current = [], None

    for w in word_tokens:
        idx   = int(np.argmax(w["avg_p"]))
        label = label_map[idx]
        score = float(w["avg_p"][idx])

        if label == "O":
            if current:
                entities.append(current)
                current = None

        elif label.startswith("B-"):
            if current:
                entities.append(current)
            current = {
                "entity": label[2:],
                "word":   text[w["start"]:w["end"]],
                "score":  score,
                "_sc": [score], "_s": w["start"], "_e": w["end"],
            }

        elif label.startswith("I-"):
            etype = label[2:]
            if current and current["entity"] == etype:
                current["word"]  = text[current["_s"]:w["end"]]
                current["_e"]    = w["end"]
                current["_sc"].append(score)
                current["score"] = float(np.mean(current["_sc"]))
            else:
                if current:
                    entities.append(current)
                current = {
                    "entity": etype,
                    "word":   text[w["start"]:w["end"]],
                    "score":  score,
                    "_sc": [score], "_s": w["start"], "_e": w["end"],
                }

    if current:
        entities.append(current)

    return [
        {"entity": e["entity"], "word": e["word"], "score": round(e["score"], 6)}
        for e in entities
    ]




```

---

## Entity Schema

The model uses a **27-label BIO scheme** covering **13 entity types** drawn directly from Algerian real estate Facebook market

| Entity | Description | Algerian Examples |
|---|---|---|
| `PROPERTY_TYPE` | Category of asset | `شقة` · `villa` · `appartement` · `terrain` · `carcasse` · `haouch` |
| `APT_CLASS` | Apartment layout | `F2` · `F3` · `F4` · `F5` · `R+1` · `Studio` |
| `TRANSACTION` | Listing intent | `للبيع` · `location` · `louer` · `خاصني` · `echange` |
| `WILAYA` | Algerian province (all 48) | `Alger` · `Oran` · `Constantine` · `16` · `ولاية وهران` |
| `CITY` | City / commune | `Bab Ezzouar` · `Sidi Yahia` · `Ain Benian` |
| `NEIGHBORHOOD` | Quarter / district / street | `باب الزوار` · `Hydra` · `Hai Yasmine` · `Télemly` |
| `PRICE` | Price in DZD, DA, or slang | `8 500 000 da` · `12M` · `950 000` · `1.5 milliards`  |
| `SURFACE` | Area in m², metres, hectares | `90m²` · `120 mètres` · `85 متر` · `2 hectares` |
| `FLOOR` | Floor level | `3ème étage` · `الطابق الثالث` · `RDC` · `R+2` |
| `PHONE` | Contact number (anonymized) | `[PHONE]` |
| `AMENITY` | Features / utilities | `garage` · `مصعد` · `piscine` · `jardin` · `بيدون` |
| `DOCUMENT` | Legal papers | `عقد` · `livret foncier` · `AADL` · `acte notarié` · `timbre` |
| `CONDITION` | Property state | `neuf` · `rénové` · `قديم` · `semi-fini` · `en construction` |

---

## Performance

Evaluated on a held-out test set of **95 posts / 596 tokenized chunks** — never seen during training or validation.

```
               precision    recall  f1-score   support

      AMENITY     0.9735    0.9971    0.9852       700
    APT_CLASS     0.9728    1.0000    0.9862       357
    CONDITION     0.9793    0.9861    0.9827       144
     DOCUMENT     0.9755    0.9848    0.9801       526
        FLOOR     0.9416    0.9928    0.9665       276
 NEIGHBORHOOD     0.8434    0.8642    0.8537        81
        PHONE     0.8101    0.9846    0.8889        65
        PRICE     0.8739    0.9336    0.9028       557
PROPERTY_TYPE     0.9869    0.9912    0.9890       682
      SURFACE     0.9259    0.9420    0.9338       517
  TRANSACTION     0.9950    0.9934    0.9942       603
       WILAYA     0.9810    0.9841    0.9825       629

    micro avg     0.9566    0.9780    0.9672      5137
    macro avg     0.9382    0.9712    0.9538      5137
 weighted avg     0.9576    0.9780    0.9675      5137
```

> **Metric:** seqeval entity-level strict match (not token-level an entity prediction counts as correct only if both the span and the label match the annotation exactly)

---

## Training Data

Sourced from the companion dataset [`81melody/algerian-realestate-ner-dataset`](https://huggingface.co/datasets/81melody/algerian-realestate-ner-dataset).

| Split | Raw posts | Tokenized chunks |
|---|---:|---:|
| Train | 1,316 | 7,138 |
| Val | 87 | 586 |
| Test | 95 | 596 |
| **Total** | **1,498** | **8,320** |

**Language distribution:** Arabic (Darja) 53% · French 34% · Mixed / Arabizi 13%  
**Listing intent:** Seller 95.4% · Buyer 4.6%

All posts were collected from public Algerian Facebook real estate groups(using Facebook API in the Apify platforl) Phone numbers are replaced with `[PHONE]` before publishing (BIO tags preserved so the model learns positional context without memorizing digits)

---

## Training Details

```yaml
base_model:        algerianDeBERTa (DeBERTa-v2)
architecture:      DebertaV2ForTokenClassification
num_labels:        27  (BIO, 13 entity types)


max_seq_len:       192  
stride:            64    


optimizer:         AdamW
peak_lr:           2e-5 
llrd_factor:       0.9   
weight_decay:      0.01
adam_eps:          1e-6
adam_beta1:        0.9
adam_beta2:        0.999
max_grad_norm:     1.0
grad_accum_steps:  2     

epochs:            20 (early stop at epoch 14, patience=5)
warmup_ratio:      0.1
schedule:          cosine with warmup


label_smoothing:   0.05
class_weighting:   inverse-frequency, capped at 10× (rare tags: CITY, WILAYA, CONDITION)
dropout:           0.1  (attention + hidden)


best_val_f1:       0.9858  
test_f1:           0.9672
test_precision:    0.9566
test_recall:       0.9780
```

### Training highlights

**Layerwise Learning Rate Decay (LLRD):** The classifier head trains at `peak_lr=2e-5`, each successive DeBERTa layer is scaled by `0.9×`, reaching `≈4.3e-6` at the embedding layer, This preserves general language representations while aggressively adapting the top layers to the NER task

**Weighted label-smoothed cross-entropy:** Rare entity tags (CITY, WILAYA, CONDITION) carry up to 10× the loss weight of frequent tags , Label smoothing (`ε=0.05`) prevents the model from becoming over-confident on the abundant `O` tag

**Sliding-window tokenization:** Posts exceeding 192 tokens are split into overlapping chunks (stride=64). Predictions from overlapping windows are reconciled at entity boundaries, ensuring long posts are fully covered without truncation

---

## Limitations

- **NEIGHBORHOOD F1=0.85:** Neighbourhood names in Algeria are highly variable in spelling across Arabic, French, and Arabizi. This entity is underrepresented in the training data (188 annotations). Performance will improve with more annotated data from underrepresented neighbourhoods
- **PRICE edge cases:** Non-standard price expressions that rely heavily on slang are occasionally missed, The model handles the most common formats reliably
- **Platform distribution:** Trained on Facebook posts — casual, informal register. May underperform on formal Arabic (MSA) or structured portal listings
- **Purely extractive:** This is a span classifier, not a generative model. It labels tokens; it does not summarise or rewrite listings

---

## Intended Use

| Use case | Notes |
|---|---|
| Structured extraction from classifieds | This is the Core use case , extract price, surface, location, type from raw posts |
| Real estate market analytics | Build price-per-m² indices by wilaya; track inventory trends |
| Lead enrichment pipelines | Enrich CRM records from social media listing text |
| Training data generation | Use model outputs as silver labels for downstream tasks |
| Algerian NLP research | Low-resource benchmark for Darja and Arabizi NER |

---

## Citation

If you use this model or the dataset in your research, please cite:

```bibtex
@misc{himeur2026algeriandeberta_ner,
  title        = {algerianDeBERTa-realestate-ner: Named Entity Recognition
                  for Algerian Real Estate Text in Darja, Arabizi, and French},
  author       = {Himeur, Ayoub},
  year         = {2026},
  publisher    = {Hugging Face},
  url          = {https://huggingface.co/81melody/algerianDeBERTa-realestate-ner},
  note         = {Fine-tuned DeBERTa-v2 on annotated
                  Algerian Facebook real estate posts, 13 entity types}
}
```

---

## License

[Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)


## Contact 
mohamed.himeur@student.unamur.be