dimeshanthoney/mbart-large-m50-singlish-sinhala

mBART-50 · Singlish → Sinhala Transliteration

Fine-tuned mBART-50 for Singlish (Romanised Sinhala) → Sinhala script transliteration using a two-phase LoRA training strategy.

Training Strategy

Phase	Dataset	Steps	LR
Phase 1	Phonetic corpus (400,000 pairs)	9,000	0.0003
Phase 2	Adhoc curated (10,000 × 10)	4,000	6e-05

LoRA config: r=32, alpha=64, dropout 0.05
Target modules: q_proj, k_proj, v_proj, out_proj

Results — IndoNLP 2025 (Test 1 + Test 2 combined)

Metric	Phase 1	Final
BLEU-char	83.4084	83.7157
WER	0.2709	0.3266
CER	0.1002	0.0986
ExactMatch	0.1784	0.0932

Inference

from transformers import MBart50TokenizerFast, MBartForConditionalGeneration

repo = 'dimeshanthoney/mbart-large-m50-singlish-sinhala'
tokenizer = MBart50TokenizerFast.from_pretrained(repo)
model = MBartForConditionalGeneration.from_pretrained(repo)

src_text = 'oya kohomada'
tokenizer.src_lang = 'en_XX'
enc = tokenizer(src_text, return_tensors='pt')

gen = model.generate(
    **enc,
    forced_bos_token_id=tokenizer.lang_code_to_id['si_LK'],
    max_length=128,
    num_beams=5
)

print(tokenizer.batch_decode(gen, skip_special_tokens=True)[0])