50M-Darija-English-v1 — Bidirectional Darija ↔ English

A 51.8M-parameter small language model that translates both ways between Moroccan Darija (الدارجة المغربية) and English. A single set of weights serves both directions; a direction-specific system prompt selects which way to translate.

Finetuned from oddadmix/50M-2048-Emhotob, a tiny Arabic base model trained from scratch.

Evaluation

Evaluated on a deterministic held-out set of 3,000 pairs (seed=42), decoded greedily (do_sample=False, no repetition penalty), scored with sacreBLEU:

Direction	sacreBLEU	chrF
Darija → English	39.72	53.22
English → Darija	43.03	52.47

Both directions are genuinely hard: Darija has no single standardized orthography, is heavily borrowed/code-switched (French/Amazigh), and appears in the data in both Arabic script and Latin (Arabizi) script. The saved weights are the best checkpoint by validation loss (eval_loss=1.032, epoch 2 of 3).

Decoding note: use plain greedy. A repetition penalty (1.2) was tested across these 50M translation models and lowered BLEU by 5–12 points.

Example translations

Real greedy-decoded outputs from the held-out set:

Darija → English

Darija input	Model output (English)
لا، عندنا تذاكر يا حبيبة	No, we have the tickets, honey
walakin hadi awal tsafira ftyyara lia o ana khayfa chwiya	But it's my first flight and I'm a little scared
أفادت التقارير أنه يوجد في المنطقة حوالي 9400 منزل بدون ماء…	It is reported that some 9400 homes in the region are without water…

(The second row shows the model handling Arabizi — Latin-script Darija — input.)

English → Darija

English input	Model output (Darija)
I'm waiting for you to deserve this	كانتسنّا فيك ت ستاهل هادشي
I dare you to object to this topic	نتحداك تعارض هاد الموضوع
but it's my first flight ever and I'm a little scared	والاكين هادي اوال تسافيرا ليا ف طييارا لاولا و انا خايفا شوييا

A larger set of 20 examples per direction (with references) is in eval_bidirectional.json.

Usage

ChatML format. Pick the system prompt for the direction you want:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "oddadmix/50M-Darija-English-v1"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16).to("cuda").eval()

SYS_TO_EN  = "You are a professional translator. Translate the Moroccan Darija text into English."
SYS_TO_DAR = "أنت مترجم محترف. ترجم النص الإنجليزي إلى الدارجة المغربية."

def translate(text: str, system: str) -> str:
    prompt = (
        f"<|im_start|>system\n{system}<|im_end|>\n"
        f"<|im_start|>user\n{text.strip()}<|im_end|>\n"
        f"<|im_start|>assistant\n"
    )
    ids = tok(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)
    if tok.bos_token_id is not None:  # training prepends BOS
        bos = torch.tensor([[tok.bos_token_id]], device=model.device)
        ids["input_ids"] = torch.cat([bos, ids["input_ids"]], dim=1)
        ids["attention_mask"] = torch.cat([torch.ones_like(bos), ids["attention_mask"]], dim=1)
    out = model.generate(**ids, max_new_tokens=256, do_sample=False,
                         eos_token_id=tok.eos_token_id, pad_token_id=tok.pad_token_id)
    return tok.decode(out[0, ids["input_ids"].size(1):], skip_special_tokens=True).strip()

print(translate("walakin hadi awal tsafira ftyyara lia o ana khayfa chwiya", SYS_TO_EN))
# → But it's my first flight and I'm a little scared
print(translate("I dare you to object to this topic", SYS_TO_DAR))
# → نتحداك تعارض هاد الموضوع

Training

• Base model: oddadmix/50M-2048-Emhotob (Llama arch, ~51.8M params)
• Dataset: oddadmix/darija_english_msa_parallel_dataset (90,104 rows; this model uses the darija and english columns). Sources: DoDA, HANTIFARAH combined, and ArabML/Skiredj parallel data.
• Method: HuggingFace Trainer, ChatML, prompt-masked cross-entropy (loss only on the assistant turn). Each row is exploded into two training examples (one per direction, ~172.2K total). Two ChatML special tokens (<|im_start|>, <|im_end|>) were added and embeddings resized.
• Hyperparameters: 3 epochs · effective batch 64 · LR 3e-4 (cosine, 5% warmup) · bf16 · max length 1024 · load_best_model_at_end on eval_loss.
• Split: 87,104 train / 3,000 deterministic held-out (seed=42), scored both directions.

Limitations

• A 50M model: expect errors on rare / technical vocabulary, proper nouns, and long or noisy inputs. Everyday conversational text is handled best.
• Darija has no standard orthography and mixes Arabic and Latin (Arabizi) script plus French/Amazigh loanwords — outputs may vary in spelling and occasionally hallucinate on out-of-domain input. The training data also contains some crawled/wiki-formatting artifacts.
• Gender is disambiguated only from context; ambiguous inputs may default one way.
• For MSA or Egyptian pairs, see the sibling models oddadmix/50M-Darija-MSA-v1, oddadmix/50M-English-MSA-v1, and oddadmix/50M-Egyptian-English-v1.

License

Apache-2.0 (model weights, inherited from the base model). Note the training dataset aggregates several community corpora — check their individual licenses for downstream use.