---
language:
- si
- en
license: apache-2.0
tags:
- transliteration
- romanization
- sinhala
- singlish
- seq2seq
- mbart
- peft
- lora
base_model: facebook/mbart-large-50-many-to-many-mmt
---

# mBART-50 · Singlish → Sinhala Transliteration

Fine-tuned **[mBART-50](https://huggingface.co/facebook/mbart-large-50-many-to-many-mmt)** for
Singlish (Romanised Sinhala) → Sinhala script transliteration using a
**two-phase LoRA training** strategy.

## Training Strategy

| Phase | Dataset | Steps | LR |
|-------|---------|-------|----|
| Phase 1 | Phonetic corpus (400,000 pairs) | 9,000 | 0.0003 |
| Phase 2 | Adhoc curated (10,000 × 10) | 4,000 | 6e-05 |

LoRA config: `r=32`, `alpha=64`, dropout `0.05`  
Target modules: `q_proj`, `k_proj`, `v_proj`, `out_proj`

## Results — IndoNLP 2025 (Test 1 + Test 2 combined)

| Metric | Phase 1 | **Final** |
|--------|---------|----------|
| BLEU-char | 83.4084 | **83.7157** |
| WER | 0.2709 | **0.3266** |
| CER | 0.1002 | **0.0986** |
| ExactMatch | 0.1784 | **0.0932** |

## Inference

```python
from transformers import MBart50TokenizerFast, MBartForConditionalGeneration

repo = 'dimeshanthoney/mbart-large-m50-singlish-sinhala'
tokenizer = MBart50TokenizerFast.from_pretrained(repo)
model = MBartForConditionalGeneration.from_pretrained(repo)

src_text = 'oya kohomada'
tokenizer.src_lang = 'en_XX'
enc = tokenizer(src_text, return_tensors='pt')

gen = model.generate(
    **enc,
    forced_bos_token_id=tokenizer.lang_code_to_id['si_LK'],
    max_length=128,
    num_beams=5
)

print(tokenizer.batch_decode(gen, skip_special_tokens=True)[0])
```