Llama-3.2-3B-ar-fa-stage3-r32

A biomedical English→Arabic and English→Persian LoRA adapter for unsloth/Llama-3.2-3B-bnb-4bit, trained jointly on 25k en→ar and 25k en→fa sentence pairs (r=32, alpha=64, all seven projection matrices, one epoch).

It scores lower on the pivot languages than the Gemma joint adapter but transfers better to unseen targets — a model inversion effect, in which intensive pivot fine-tuning appears to overwrite Gemma's broader multilingual representations while Llama retains more generalisable cross-lingual priors. It is therefore the right choice for few-shot in-context transfer, where it gives the best few-shot result in the study: Dari 34.84 CHrF++ under 3-shot ICL.

This is a LoRA adapter, not a merged model: the base model is required at inference time. It must not be merged with the Gemma adapters — the tensor shapes will not align.

Scores

CHrF++ (sacrebleu, --chrf-word-order 2, 0–100) on the 971-sentence TICO-19 development split.

Direction	dev
en→ar	28.87
en→fa	31.44

Apply Unicode normalisation to hypotheses and references before scoring: CAMeL Tools for Arabic, Hazm for Persian and Dari.

Prompt template

Reproduce it exactly; quality degrades noticeably with other phrasings. {language} takes the English name of the target language, such as Arabic, Persian, or Dari.

Translate the below text from English to {language}. Only output the final translation in {language}; do not include any additional text.

### English text:
{source}

### {language} translation:

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE = "unsloth/Llama-3.2-3B-bnb-4bit"
ADAPTER = "ArashEslam/Llama-3.2-3B-ar-fa-stage3-r32"
LANGUAGE = "Persian"

PROMPT = (
    "Translate the below text from English to {language}. Only output the final "
    "translation in {language}; do not include any additional text.\n\n"
    "### English text:\n{source}\n\n"
    "### {language} translation:\n"
)

tokenizer = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(BASE, device_map="auto", torch_dtype=torch.bfloat16)
model = PeftModel.from_pretrained(model, ADAPTER)
model.eval()

source = "Wash your hands with soap and water for at least 20 seconds."
inputs = tokenizer(
    PROMPT.format(language=LANGUAGE, source=source),
    return_tensors="pt", truncation=True, max_length=256,
).to(model.device)

with torch.no_grad():
    output = model.generate(**inputs, max_new_tokens=128, do_sample=False)

decoded = tokenizer.decode(output[0], skip_special_tokens=True)
print(decoded.split(f"### {LANGUAGE} translation:")[-1].strip())

Greedy decoding is required to reproduce the reported scores. The model continues past the prompt, so splitting on the ### {language} translation: marker and keeping the final segment is not optional. For few-shot transfer to an unseen target language, prepend FLORES-200 demonstrations in the same format.

⚠ Intended use and safety

This is a research artifact. It must not be used for clinical decision-making, patient communication, or any other safety-critical purpose without qualified human review. Translation errors in healthcare lead to misunderstanding, delayed treatment, and direct patient harm.

Numbers, dosages, units, and dates are not reliably preserved. Do not use this model where a numeric value carries clinical meaning unless every figure is verified against the source.

It is a general-purpose LLM adapted with LoRA, not a system engineered for translation, and is subject to hallucination, prompt sensitivity, and fluent-but-wrong output — the most dangerous failure mode in a medical register, because the output looks correct. Scores are measured on TICO-19 public-health text; performance on specialised clinical, pharmaceutical, or diagnostic terminology is unmeasured and likely worse. Professional medical translator evaluation is required before any deployment.

Links

• Paper: arXiv:2607.22300
• Code, merging scripts, and the other released adapters: https://github.com/arasheslamii/biomed-mt-arabic-script

Built with Llama. Use of this adapter is subject to the Llama 3.2 Community License.

Citation

@article{alabdullah2026biomedical,
  title   = {Biomedical Machine Translation for Low-Resource Arabic-Script Languages
             via Cross-Lingual Transfer and {LoRA} Adapter Merging},
  author  = {Alabdullah, Abdullah and Eslamighayour, Arash and
             Harbalioglu, Sarp and Han, Lifeng},
  journal = {arXiv preprint arXiv:2607.22300},
  year    = {2026},
  url     = {https://arxiv.org/abs/2607.22300}
}