README.md

metadata

license: cc-by-nc-4.0
library_name: transformers
pipeline_tag: translation
base_model: facebook/nllb-200-distilled-600M
language:
  - zh
  - ami
  - bnn
  - ckv
  - dru
  - pwn
  - pyu
  - ssf
  - sxr
  - szy
  - tao
  - tay
  - trv
  - tsu
  - xnb
  - xsy
tags:
  - translation
  - nllb
  - nllb-200
  - low-resource
  - endangered-languages
  - formosan-languages
  - sentencepiece
  - private-no-bible
metrics:
  - bleu
  - chrf2
  - ter
model-index:
  - name: nllb200-formosan-zh-spm8k
    results:
      - task:
          name: Machine Translation
          type: translation
        dataset:
          name: FormosanBank Traditional Chinese private no-Bible hard split
          type: custom
        metrics:
          - name: BLEU
            type: bleu
            value: 10.5767
            args:
              direction: f2zh
              samples: 61384
              tokenize: zh
          - name: chrF2
            type: chrf2
            value: 12.3912
            args:
              direction: f2zh
              samples: 61384
          - name: TER
            type: ter
            value: 115.7521
            args:
              direction: f2zh
              samples: 61384

nllb200-formosan-zh-spm8k

Base model: facebook/nllb-200-distilled-600M
Direction: Formosan -> Traditional Chinese
Companion model: FormosanBank/nllb200-zh-formosan-spm8k
Release: private no-Bible SPM8k flight 20260712-232900, validation-selected step 270,000

This directional checkpoint replaces the earlier release with the strongest private_no_bible model from the fully rebuilt FormosanBank MT pipeline. It uses an 8,192-piece Formosan-aware SentencePiece extension and explicit direction, source-language, source-domain, and dialect control tags.

Supported Languages

Language	NLLB code
Traditional Chinese	zho_Hant
Amis	ami_Latn
Bunun	bnn_Latn
Kavalan	ckv_Latn
Rukai	dru_Latn
Paiwan	pwn_Latn
Puyuma	pyu_Latn
Thao	ssf_Latn
Saaroa	sxr_Latn
Sakizaya	szy_Latn
Tao / Yami	tao_Latn
Atayal	tay_Latn
Seediq	trv_Latn
Tsou	tsu_Latn
Kanakanavu	xnb_Latn
Saisiyat	xsy_Latn

Input Format

Prefix every source with:

<to_zh> <src_LANG> <dom_BUCKET> <dialect_DIALECT>

Example:

<to_zh> <src_ami> <dom_unknown> <dialect_default> Pa'araw cingra to demak nira.

Use <dom_unknown> and <dialect_default> when metadata is unavailable.

Usage

Use the slow NllbTokenizer (use_fast=False with AutoTokenizer). These checkpoints were trained with transformers==4.56.1; fast-tokenizer added-token IDs can differ from the slow tokenizer IDs used in training. NLLB generation must start with the tokenizer EOS ID and force the target-language BOS ID.

import torch
from transformers import AutoModelForSeq2SeqLM, NllbTokenizer

model_id = "FormosanBank/nllb200-formosan-zh-spm8k"
tokenizer = NllbTokenizer.from_pretrained(model_id)
model = AutoModelForSeq2SeqLM.from_pretrained(model_id)
model.to("cuda" if torch.cuda.is_available() else "cpu")

FORMOSAN_TO_LID = {
    "ami": "ami_Latn", "bnn": "bnn_Latn", "ckv": "ckv_Latn", "dru": "dru_Latn",
    "pwn": "pwn_Latn", "pyu": "pyu_Latn", "ssf": "ssf_Latn", "sxr": "sxr_Latn",
    "szy": "szy_Latn", "tao": "tao_Latn", "tay": "tay_Latn", "trv": "trv_Latn",
    "tsu": "tsu_Latn", "xnb": "xnb_Latn", "xsy": "xsy_Latn",
}

def translate_formosan_to_chinese(text: str, lang_code: str, source_bucket: str = "unknown", dialect: str = "default") -> str:
    tokenizer.src_lang = FORMOSAN_TO_LID[lang_code]
    prompt = f"<to_zh> <src_{lang_code}> <dom_{source_bucket}> <dialect_{dialect}> {text}"
    inputs = tokenizer(prompt, return_tensors="pt", truncation=True, max_length=384).to(model.device)
    outputs = model.generate(
        **inputs,
        forced_bos_token_id=tokenizer.convert_tokens_to_ids("zho_Hant"),
        decoder_start_token_id=tokenizer.eos_token_id,
        max_new_tokens=128,
        num_beams=4,
        no_repeat_ngram_size=3,
        repetition_penalty=1.15,
        early_stopping=True,
    )
    return tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]

print(translate_formosan_to_chinese("Pa'araw cingra to demak nira.", "ami"))

Checkpoint Selection

The published checkpoint was selected only on validation chrF2, not on the hard test set.

Selection step	Validation samples	Validation loss	Perplexity	BLEU	chrF2	TER
270,000	1,920	2.4207	11.25	23.58	22.94	103.94

Validation generation sampled 128 rows per Formosan language every 10,000 updates. The full hard test was evaluated only after selection.

Training Setup

Setting	Value
Corpus	private_no_bible (Traditional Chinese)
Base model	facebook/nllb-200-distilled-600M
Maximum updates	300,000
Published best step	270,000
Microbatch / accumulation	16 / 4
Effective batch	64
Maximum length	384
Learning rate	2e-05
Precision	bf16
Easy-source weight	0.05
Language sampling alpha	0.5
Metadata control tags	enabled and validated as single tokenizer IDs

Corpus and Split Integrity

Total	Train	Test	Validate	Minimum per-language test	Minimum per-language validate
791,330	708,496	61,384	21,450	7.5%	2.5%

The exact Formosan-Taiwan-Bible-Society-Bibles repository is excluded. Lexical entries are train-only. Independent validation found zero normalized source, target, or pair overlap; zero punctuation/spacing skeleton overlap; and zero one-edit source or target conflicts across train and evaluation. Connected similarity groups are assigned as units so variants cannot be split independently merely because they are not exact duplicates.

All hard-test references in this Chinese corpus are original rather than pivot-generated.

Hard-Test Results

SacreBLEU was computed with zh tokenization; chrF uses beta 2; TER is lower-is-better.

Direction	Samples	BLEU	chrF2	TER	Exact match	Empty output
Formosan -> Traditional Chinese	61,384	10.58	12.39	115.75	1.40%	0.00%

Per-Language Results

Language	Code	Samples	BLEU	chrF2	TER
Amis	ami_Latn	10,929	7.91	10.07	115.26
Bunun	bnn_Latn	5,450	9.24	10.91	120.58
Kavalan	ckv_Latn	2,852	15.76	16.77	108.05
Rukai	dru_Latn	5,524	8.14	10.05	127.33
Paiwan	pwn_Latn	5,519	8.81	10.99	111.15
Puyuma	pyu_Latn	4,105	13.10	15.12	107.76
Thao	ssf_Latn	1,670	14.87	16.29	110.56
Saaroa	sxr_Latn	1,539	8.57	11.04	109.41
Sakizaya	szy_Latn	2,147	12.58	15.79	110.10
Tao / Yami	tao_Latn	1,802	8.32	11.30	109.37
Atayal	tay_Latn	6,395	8.92	10.24	125.92
Seediq	trv_Latn	6,689	17.66	18.01	107.97
Tsou	tsu_Latn	1,889	8.00	10.40	115.47
Kanakanavu	xnb_Latn	2,927	17.48	18.51	111.07
Saisiyat	xsy_Latn	1,947	9.94	12.25	125.58

Full source-bucket and length-bin breakdowns are in eval/metrics.json.

Intended Use

• Research, teaching, and prototyping for Formosan-language machine translation.
• Draft translation assistance where knowledgeable speakers can review the output.
• Comparative low-resource MT evaluation on the documented leakage-controlled split.

Limitations

• Output may be incorrect, ungrammatical, incomplete, or culturally inappropriate.
• Formosan generation is draft-only and requires speaker review.
• Aggregate scores across 15 languages conceal substantial per-language variation.
• This model is unsuitable for legal, medical, safety-critical, or authoritative community-facing use without expert review.
• Hard-split scores are not directly comparable with earlier evaluations that allowed stronger train-test similarity.

License

Released under cc-by-nc-4.0. Underlying corpus sources may impose additional restrictions. Confirm the rights needed for your use case.

Citation

@misc{formosanbank_nllb200_formosan_zh_spm8k_2026,
  title  = {nllb200-formosan-zh-spm8k: Directional NLLB-200 MT on the FormosanBank private no-Bible corpus},
  author = {FormosanBank contributors},
  year   = {2026},
  url    = {https://huggingface.co/FormosanBank/nllb200-formosan-zh-spm8k}
}