Replace with private no-Bible SPM8k best checkpoint (20260712-232900)

Browse files

Files changed (10) hide show

.gitattributes +1 -34
README.md +87 -70
added_tokens.json +152 -80
config.json +1 -1
eval/metrics.json +432 -147
experiment_metadata.json +181 -2
model.safetensors +2 -2
sentencepiece.bpe.model +2 -2
special_tokens_map.json +73 -1
tokenizer_config.json +728 -80

.gitattributes CHANGED Viewed

@@ -1,35 +1,2 @@
-*.7z filter=lfs diff=lfs merge=lfs -text
-*.arrow filter=lfs diff=lfs merge=lfs -text
-*.bin filter=lfs diff=lfs merge=lfs -text
-*.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
-*.ftz filter=lfs diff=lfs merge=lfs -text
-*.gz filter=lfs diff=lfs merge=lfs -text
-*.h5 filter=lfs diff=lfs merge=lfs -text
-*.joblib filter=lfs diff=lfs merge=lfs -text
-*.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
-*.model filter=lfs diff=lfs merge=lfs -text
-*.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
-*.onnx filter=lfs diff=lfs merge=lfs -text
-*.ot filter=lfs diff=lfs merge=lfs -text
-*.parquet filter=lfs diff=lfs merge=lfs -text
-*.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
-*.pt filter=lfs diff=lfs merge=lfs -text
-*.pth filter=lfs diff=lfs merge=lfs -text
-*.rar filter=lfs diff=lfs merge=lfs -text
 *.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
-*.tflite filter=lfs diff=lfs merge=lfs -text
-*.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
-*.xz filter=lfs diff=lfs merge=lfs -text
-*.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text


























1	*.safetensors filter=lfs diff=lfs merge=lfs -text
2	+ *.model filter=lfs diff=lfs merge=lfs -text

README.md CHANGED Viewed

@@ -27,10 +27,8 @@ tags:
 - low-resource
 - endangered-languages
 - formosan-languages
-- seq2seq
-- encoder-decoder
 - sentencepiece
-- traditional-chinese
 metrics:
 - bleu
 - chrf2
@@ -42,38 +40,40 @@ model-index:
       name: Machine Translation
       type: translation
     dataset:
-      name: FormosanBank Chinese Parallel Corpus, leakage-controlled in-domain hard split
       type: custom
     metrics:
     - name: BLEU
       type: bleu
-      value: 9.79
       args:
-        direction: formosan-to-zho_Hant
-        samples: 37435
         tokenize: zh
     - name: chrF2
       type: chrf2
-      value: 11.77
       args:
-        direction: formosan-to-zho_Hant
-        samples: 37435
     - name: TER
       type: ter
-      value: 109.29
       args:
-        direction: formosan-to-zho_Hant
-        samples: 37435
 ---
 # nllb200-formosan-zh-spm8k
-**Repo:** `FormosanBank/nllb200-formosan-zh-spm8k`
 **Base model:** [`facebook/nllb-200-distilled-600M`](https://huggingface.co/facebook/nllb-200-distilled-600M)
 **Direction:** **Formosan -> Traditional Chinese**
-**Companion reverse-direction model:** [`FormosanBank/nllb200-zh-formosan-spm8k`](https://huggingface.co/FormosanBank/nllb200-zh-formosan-spm8k)
-This is a directional NLLB-200 distilled 600M checkpoint for FormosanBank machine translation. It uses an 8k SentencePiece vocabulary extension plus FormosanBank metadata/control tags. Use the companion model for the reverse direction.
 ## Supported Languages
@@ -98,21 +98,21 @@ This is a directional NLLB-200 distilled 600M checkpoint for FormosanBank machin
 ## Input Format
-This model was trained and evaluated with metadata control tags. Prefix the source text in one of the supported Formosan languages with:
 `<to_zh> <src_LANG> <dom_BUCKET> <dialect_DIALECT>`
-Example with unknown metadata:
 `<to_zh> <src_ami> <dom_unknown> <dialect_default> Pa'araw cingra to demak nira.`
-If source bucket or dialect is unknown, use `<dom_unknown>` and `<dialect_default>`. If you know the training source bucket or dialect, using the matching tag may improve quality.
 ## Usage
-Tested with `transformers` 4.56.x. Use the slow `NllbTokenizer` or `AutoTokenizer.from_pretrained(model_id, use_fast=False)`. In `transformers` 4.56.x, fast-tokenizer added-token IDs can differ from the slow tokenizer IDs used during training.
-For NLLB generation, keep `decoder_start_token_id=tokenizer.eos_token_id` and set `forced_bos_token_id` to the target language ID.
 ```python
 import torch
@@ -142,7 +142,6 @@ def translate_formosan_to_chinese(text: str, lang_code: str, source_bucket: str
         num_beams=4,
         no_repeat_ngram_size=3,
         repetition_penalty=1.15,
-        length_penalty=1.0,
         early_stopping=True,
     )
     return tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]
@@ -150,83 +149,101 @@ def translate_formosan_to_chinese(text: str, lang_code: str, source_bucket: str
 print(translate_formosan_to_chinese("Pa'araw cingra to demak nira.", "ami"))
 ```
 ## Training Setup
 | Setting | Value |
 |---|---|
-| Corpus | FormosanBank Chinese Parallel Corpus, leakage-controlled in-domain hard split |
-| Direction | `f2zh` |
 | Base model | `facebook/nllb-200-distilled-600M` |
-| Tokenizer | 8k Formosan SentencePiece extension |
-| Steps | 300,000 |
-| Batch size | 16 |
-| Gradient accumulation | 4 |
-| Effective batch size | 64 |
-| Max sequence length | 384 |
-| Learning rate | 2e-05 |
-| Warmup steps | 4,000 |
 | Precision | `bf16` |
-| Label smoothing | 0.1 |
 | Easy-source weight | 0.05 |
 | Language sampling alpha | 0.5 |
-| Metadata tags | enabled and validated as single tokenizer IDs |
-This repo publishes the final 300k-step checkpoint because it scored highest on the held-out hard test set among the evaluated final/best checkpoints.
-## Evaluation
-Evaluation used the held-out `in_domain_hard` test split with no normalized source, target, or pair overlap against train. These scores are intentionally lower than leaky or near-duplicate splits and are intended as a harder MT benchmark.
-### Global Metrics
-| Direction | Samples | BLEU | chrF2 | TER |
-|---|---:|---:|---:|---:|
-| Formosan -> Traditional Chinese | 37,435 | 9.79 | 11.77 | 109.29 |
-### Per-Language Metrics
 | Language | Code | Samples | BLEU | chrF2 | TER |
 |---|---:|---:|---:|---:|---:|
-| Amis | `ami_Latn` | 4,866 | 9.68 | 11.26 | 107.56 |
-| Bunun | `bnn_Latn` | 3,223 | 9.44 | 10.52 | 105.04 |
-| Kavalan | `ckv_Latn` | 1,832 | 14.27 | 15.11 | 108.67 |
-| Rukai | `dru_Latn` | 3,846 | 5.57 | 8.57 | 108.67 |
-| Paiwan | `pwn_Latn` | 3,221 | 7.86 | 9.95 | 106.33 |
-| Puyuma | `pyu_Latn` | 2,225 | 15.77 | 16.93 | 124.55 |
-| Thao | `ssf_Latn` | 1,180 | 14.73 | 16.36 | 106.69 |
-| Saaroa | `sxr_Latn` | 1,106 | 9.17 | 12.79 | 108.07 |
-| Sakizaya | `szy_Latn` | 1,601 | 11.42 | 14.83 | 104.58 |
-| Tao / Yami | `tao_Latn` | 1,455 | 6.93 | 10.38 | 110.85 |
-| Atayal | `tay_Latn` | 4,293 | 6.83 | 9.07 | 108.89 |
-| Seediq | `trv_Latn` | 4,573 | 10.09 | 11.96 | 109.35 |
-| Tsou | `tsu_Latn` | 1,250 | 8.56 | 11.42 | 126.84 |
-| Kanakanavu | `xnb_Latn` | 1,552 | 12.96 | 15.19 | 107.91 |
-| Saisiyat | `xsy_Latn` | 1,212 | 14.45 | 16.32 | 106.80 |
-Full source-bucket and length-bin breakdowns are available in [`eval/metrics.json`](eval/metrics.json).
 ## Intended Use
-- Research, teaching, and prototyping for Formosan-language MT.
-- Draft translation assistance where review by knowledgeable speakers is available.
-- Comparative evaluation of low-resource MT methods on leakage-controlled FormosanBank splits.
 ## Limitations
-- Outputs can be incorrect, ungrammatical, incomplete, or culturally inappropriate.
-- Generation into Formosan languages is especially difficult and should be treated as draft-only.
-- This model is not suitable for legal, medical, safety-critical, or authoritative community-facing use without expert review.
-- Evaluation uses a hard split; BLEU should not be compared directly to older leaky or near-duplicate split results.
 ## License
-Released under `cc-by-nc-4.0`. Some underlying corpus sources may carry additional restrictions. Use this model only for non-commercial research and educational purposes unless you have confirmed broader rights for your use case.
 ## Citation
 ```bibtex
-@misc{formosanbank_nllb200_formosan_zh_spm8k,
-  title  = {nllb200-formosan-zh-spm8k: Directional NLLB-200 MT for the FormosanBank Chinese Parallel Corpus, leakage-controlled in-domain hard split},
   author = {FormosanBank contributors},
   year   = {2026},
   url    = {https://huggingface.co/FormosanBank/nllb200-formosan-zh-spm8k}

 - low-resource
 - endangered-languages
 - formosan-languages
 - sentencepiece
+- private-no-bible
 metrics:
 - bleu
 - chrf2
       name: Machine Translation
       type: translation
     dataset:
+      name: FormosanBank Traditional Chinese private no-Bible hard split
       type: custom
     metrics:
     - name: BLEU
       type: bleu
+      value: 10.5767
       args:
+        direction: f2zh
+        samples: 61384
         tokenize: zh
     - name: chrF2
       type: chrf2
+      value: 12.3912
       args:
+        direction: f2zh
+        samples: 61384
     - name: TER
       type: ter
+      value: 115.7521
       args:
+        direction: f2zh
+        samples: 61384
 ---
 # nllb200-formosan-zh-spm8k
 **Base model:** [`facebook/nllb-200-distilled-600M`](https://huggingface.co/facebook/nllb-200-distilled-600M)
 **Direction:** **Formosan -> Traditional Chinese**
+**Companion model:** [`FormosanBank/nllb200-zh-formosan-spm8k`](https://huggingface.co/FormosanBank/nllb200-zh-formosan-spm8k)
+**Release:** private no-Bible SPM8k flight `20260712-232900`, validation-selected step `270,000`
+This directional checkpoint replaces the earlier release with the strongest `private_no_bible` model from the
+fully rebuilt FormosanBank MT pipeline. It uses an 8,192-piece Formosan-aware SentencePiece extension and explicit
+direction, source-language, source-domain, and dialect control tags.
 ## Supported Languages
 ## Input Format
+Prefix every source with:
 `<to_zh> <src_LANG> <dom_BUCKET> <dialect_DIALECT>`
+Example:
 `<to_zh> <src_ami> <dom_unknown> <dialect_default> Pa'araw cingra to demak nira.`
+Use `<dom_unknown>` and `<dialect_default>` when metadata is unavailable.
 ## Usage
+Use the slow `NllbTokenizer` (`use_fast=False` with `AutoTokenizer`). These checkpoints were trained with
+`transformers==4.56.1`; fast-tokenizer added-token IDs can differ from the slow tokenizer IDs used in training.
+NLLB generation must start with the tokenizer EOS ID and force the target-language BOS ID.
 ```python
 import torch
         num_beams=4,
         no_repeat_ngram_size=3,
         repetition_penalty=1.15,
         early_stopping=True,
     )
     return tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]
 print(translate_formosan_to_chinese("Pa'araw cingra to demak nira.", "ami"))
 ```
+## Checkpoint Selection
+The published checkpoint was selected **only on validation chrF2**, not on the hard test set.
+| Selection step | Validation samples | Validation loss | Perplexity | BLEU | chrF2 | TER |
+|---:|---:|---:|---:|---:|---:|---:|
+| 270,000 | 1,920 | 2.4207 | 11.25 | 23.58 | 22.94 | 103.94 |
+Validation generation sampled 128 rows per Formosan language every 10,000 updates. The full hard test was evaluated
+only after selection.
 ## Training Setup
 | Setting | Value |
 |---|---|
+| Corpus | `private_no_bible` (Traditional Chinese) |
 | Base model | `facebook/nllb-200-distilled-600M` |
+| Maximum updates | 300,000 |
+| Published best step | 270,000 |
+| Microbatch / accumulation | 16 / 4 |
+| Effective batch | 64 |
+| Maximum length | 384 |
+| Learning rate | `2e-05` |
 | Precision | `bf16` |
 | Easy-source weight | 0.05 |
 | Language sampling alpha | 0.5 |
+| Metadata control tags | enabled and validated as single tokenizer IDs |
+## Corpus and Split Integrity
+| Total | Train | Test | Validate | Minimum per-language test | Minimum per-language validate |
+|---:|---:|---:|---:|---:|---:|
+| 791,330 | 708,496 | 61,384 | 21,450 | 7.5% | 2.5% |
+The exact `Formosan-Taiwan-Bible-Society-Bibles` repository is excluded. Lexical entries are train-only. Independent
+validation found zero normalized source, target, or pair overlap; zero punctuation/spacing skeleton overlap; and zero
+one-edit source or target conflicts across train and evaluation. Connected similarity groups are assigned as units so
+variants cannot be split independently merely because they are not exact duplicates.
+All hard-test references in this Chinese corpus are original rather than pivot-generated.
+## Hard-Test Results
+SacreBLEU was computed with `zh` tokenization; chrF uses beta 2; TER is lower-is-better.
+| Direction | Samples | BLEU | chrF2 | TER | Exact match | Empty output |
+|---|---:|---:|---:|---:|---:|---:|
+| Formosan -> Traditional Chinese | 61,384 | 10.58 | 12.39 | 115.75 | 1.40% | 0.00% |
+### Per-Language Results
 | Language | Code | Samples | BLEU | chrF2 | TER |
 |---|---:|---:|---:|---:|---:|
+| Amis | `ami_Latn` | 10,929 | 7.91 | 10.07 | 115.26 |
+| Bunun | `bnn_Latn` | 5,450 | 9.24 | 10.91 | 120.58 |
+| Kavalan | `ckv_Latn` | 2,852 | 15.76 | 16.77 | 108.05 |
+| Rukai | `dru_Latn` | 5,524 | 8.14 | 10.05 | 127.33 |
+| Paiwan | `pwn_Latn` | 5,519 | 8.81 | 10.99 | 111.15 |
+| Puyuma | `pyu_Latn` | 4,105 | 13.10 | 15.12 | 107.76 |
+| Thao | `ssf_Latn` | 1,670 | 14.87 | 16.29 | 110.56 |
+| Saaroa | `sxr_Latn` | 1,539 | 8.57 | 11.04 | 109.41 |
+| Sakizaya | `szy_Latn` | 2,147 | 12.58 | 15.79 | 110.10 |
+| Tao / Yami | `tao_Latn` | 1,802 | 8.32 | 11.30 | 109.37 |
+| Atayal | `tay_Latn` | 6,395 | 8.92 | 10.24 | 125.92 |
+| Seediq | `trv_Latn` | 6,689 | 17.66 | 18.01 | 107.97 |
+| Tsou | `tsu_Latn` | 1,889 | 8.00 | 10.40 | 115.47 |
+| Kanakanavu | `xnb_Latn` | 2,927 | 17.48 | 18.51 | 111.07 |
+| Saisiyat | `xsy_Latn` | 1,947 | 9.94 | 12.25 | 125.58 |
+Full source-bucket and length-bin breakdowns are in [`eval/metrics.json`](eval/metrics.json).
 ## Intended Use
+- Research, teaching, and prototyping for Formosan-language machine translation.
+- Draft translation assistance where knowledgeable speakers can review the output.
+- Comparative low-resource MT evaluation on the documented leakage-controlled split.
 ## Limitations
+- Output may be incorrect, ungrammatical, incomplete, or culturally inappropriate.
+- Formosan generation is draft-only and requires speaker review.
+- Aggregate scores across 15 languages conceal substantial per-language variation.
+- This model is unsuitable for legal, medical, safety-critical, or authoritative community-facing use without expert review.
+- Hard-split scores are not directly comparable with earlier evaluations that allowed stronger train-test similarity.
 ## License
+Released under `cc-by-nc-4.0`. Underlying corpus sources may impose additional restrictions. Confirm the rights needed
+for your use case.
 ## Citation
 ```bibtex
+@misc{formosanbank_nllb200_formosan_zh_spm8k_2026,
+  title  = {nllb200-formosan-zh-spm8k: Directional NLLB-200 MT on the FormosanBank private no-Bible corpus},
   author = {FormosanBank contributors},
   year   = {2026},
   url    = {https://huggingface.co/FormosanBank/nllb200-formosan-zh-spm8k}

added_tokens.json CHANGED Viewed

@@ -12,86 +12,158 @@
   "<dialect_hengchun>": 259649,
   "<dialect_jianhe>": 259650,
   "<dialect_junqun>": 259651,
-  "<dialect_kaqun>": 259652,
-  "<dialect_luanqun>": 259653,
-  "<dialect_malan>": 259654,
-  "<dialect_maolin>": 259655,
-  "<dialect_nanwang>": 259656,
-  "<dialect_northern>": 259657,
-  "<dialect_sekolik>": 259658,
-  "<dialect_southern>": 259659,
-  "<dialect_tanqun>": 259660,
-  "<dialect_tegudaya>": 259661,
-  "<dialect_truku>": 259662,
-  "<dialect_unknown>": 259663,
-  "<dialect_wanda>": 259664,
-  "<dialect_wanshan>": 259665,
-  "<dialect_wenshui>": 259666,
-  "<dialect_wutai>": 259667,
-  "<dialect_xiqun>": 259668,
-  "<dialect_xiuguluan>": 259669,
-  "<dialect_yilan_zeaol>": 259670,
-  "<dialect_zeaol>": 259671,
-  "<dialect_zhiben>": 259672,
-  "<dialect_zhuoqun>": 259673,
-  "<dom_classroom_context>": 259674,
-  "<dom_culture>": 259675,
-  "<dom_dictionary>": 259676,
-  "<dom_essays>": 259677,
-  "<dom_formosan_100_paiwan_texts>": 259678,
-  "<dom_formosan_amis_myths_and_customs>": 259679,
-  "<dom_formosan_epark>": 259680,
-  "<dom_formosan_gitbook_translations>": 259681,
-  "<dom_formosan_old_texts>": 259682,
-  "<dom_formosan_paiwanstories>": 259683,
-  "<dom_formosan_rik_bunun>": 259684,
-  "<dom_formosan_seals>": 259685,
-  "<dom_formosan_wilang_yutas_videos>": 259686,
-  "<dom_formosan_yeddas_blog>": 259687,
-  "<dom_formosan_zheng_data>": 259688,
-  "<dom_learning_vocab>": 259689,
-  "<dom_nine_level>": 259690,
-  "<dom_ntu>": 259691,
-  "<dom_picture_book>": 259692,
-  "<dom_picture_story>": 259693,
-  "<dom_presidential_apology>": 259694,
-  "<dom_reading_writing>": 259695,
-  "<dom_unknown>": 259696,
-  "<dom_youtube>": 259697,
-  "<src_ami>": 259698,
-  "<src_bnn>": 259699,
-  "<src_ckv>": 259700,
-  "<src_dru>": 259701,
-  "<src_eng>": 259702,
-  "<src_pwn>": 259703,
-  "<src_pyu>": 259704,
-  "<src_ssf>": 259705,
-  "<src_sxr>": 259706,
-  "<src_szy>": 259707,
-  "<src_tao>": 259708,
-  "<src_tay>": 259709,
-  "<src_trv>": 259710,
-  "<src_tsu>": 259711,
-  "<src_xnb>": 259712,
-  "<src_xsy>": 259713,
-  "<src_zh>": 259714,
-  "<to_ami>": 259715,
-  "<to_bnn>": 259716,
-  "<to_ckv>": 259717,
-  "<to_dru>": 259718,
-  "<to_eng>": 259719,
-  "<to_pwn>": 259720,
-  "<to_pyu>": 259721,
-  "<to_ssf>": 259722,
-  "<to_sxr>": 259723,
-  "<to_szy>": 259724,
-  "<to_tao>": 259725,
-  "<to_tay>": 259726,
-  "<to_trv>": 259727,
-  "<to_tsu>": 259728,
-  "<to_xnb>": 259729,
-  "<to_xsy>": 259730,
-  "<to_zh>": 259731,
   "ami_Latn": 259624,
   "bnn_Latn": 259625,
   "ckv_Latn": 259626,

   "<dialect_hengchun>": 259649,
   "<dialect_jianhe>": 259650,
   "<dialect_junqun>": 259651,
+  "<dialect_kanakanavu>": 259652,
+  "<dialect_kaqun>": 259653,
+  "<dialect_kavalan>": 259654,
+  "<dialect_luanqun>": 259655,
+  "<dialect_malan>": 259656,
+  "<dialect_maolin>": 259657,
+  "<dialect_nanwang>": 259658,
+  "<dialect_northern>": 259659,
+  "<dialect_saaroa>": 259660,
+  "<dialect_saisiyat>": 259661,
+  "<dialect_sakizaya>": 259662,
+  "<dialect_sekolik>": 259663,
+  "<dialect_southern>": 259664,
+  "<dialect_tanqun>": 259665,
+  "<dialect_tegudaya>": 259666,
+  "<dialect_thao>": 259667,
+  "<dialect_truku>": 259668,
+  "<dialect_tsou>": 259669,
+  "<dialect_unknown>": 259670,
+  "<dialect_wanda>": 259671,
+  "<dialect_wanshan>": 259672,
+  "<dialect_wenshui>": 259673,
+  "<dialect_wutai>": 259674,
+  "<dialect_xiqun>": 259675,
+  "<dialect_xiuguluan>": 259676,
+  "<dialect_yami>": 259677,
+  "<dialect_yilanzeaol>": 259678,
+  "<dialect_zeaol>": 259679,
+  "<dialect_zhiben>": 259680,
+  "<dialect_zhuoqun>": 259681,
+  "<dom_classroom_context>": 259682,
+  "<dom_culture>": 259683,
+  "<dom_dictionary>": 259684,
+  "<dom_essays>": 259685,
+  "<dom_ethnography_journal_ritual_song>": 259686,
+  "<dom_formosan_100_paiwan_texts>": 259687,
+  "<dom_formosan_academia_sinica_oral_legends>": 259688,
+  "<dom_formosan_amis_adversative_constructions>": 259689,
+  "<dom_formosan_amis_kavalan_lin_interrogative_verbs>": 259690,
+  "<dom_formosan_amis_kuo_sung_comparative_constructions>": 259691,
+  "<dom_formosan_amis_lifok_dongi_saopo_kimakimad>": 259692,
+  "<dom_formosan_amis_myths_and_customs>": 259693,
+  "<dom_formosan_amis_pa_verbs>": 259694,
+  "<dom_formosan_amis_serial_verb_constructions>": 259695,
+  "<dom_formosan_amis_tung_chiou>": 259696,
+  "<dom_formosan_asai_sedik_language>": 259697,
+  "<dom_formosan_bunun_debusser_dissertation>": 259698,
+  "<dom_formosan_bunun_moriguchi_northern_texts>": 259699,
+  "<dom_formosan_bunun_topic_focus>": 259700,
+  "<dom_formosan_chen_fukuda_relabeling_ergative>": 259701,
+  "<dom_formosan_chen_fukuda_three_ways_steal>": 259702,
+  "<dom_formosan_cip_atayal_grammar_overview>": 259703,
+  "<dom_formosan_dean_johnson_year_clouds_smangus>": 259704,
+  "<dom_formosan_dorinda_tsai_hsiu_liu_neutral_imperfect>": 259705,
+  "<dom_formosan_egerod_origin_headhunting_atayal_text_v>": 259706,
+  "<dom_formosan_egerod_soren_word_order_word_classes_at>": 259707,
+  "<dom_formosan_elizabeth_zeitoun_chen_huei_wu_overview>": 259708,
+  "<dom_formosan_epark>": 259709,
+  "<dom_formosan_ferrell_taiwan_aboriginal_groups_proble>": 259710,
+  "<dom_formosan_gitbook_translations>": 259711,
+  "<dom_formosan_glosbe>": 259712,
+  "<dom_formosan_hala_saku_la_videos>": 259713,
+  "<dom_formosan_holmer_seediq>": 259714,
+  "<dom_formosan_huang_grammaticalization_squliq_atayal>": 259715,
+  "<dom_formosan_huang_mei_chin_atayal_reference_grammar>": 259716,
+  "<dom_formosan_huteson_rukai_survey>": 259717,
+  "<dom_formosan_ilrdf_42_language_practice_word_lists>": 259718,
+  "<dom_formosan_ilrdf_tousvusvutu_kita>": 259719,
+  "<dom_formosan_indigenous_language_literary_awards>": 259720,
+  "<dom_formosan_kanakanavu_texts>": 259721,
+  "<dom_formosan_kavalan_zhang_reference_grammar>": 259722,
+  "<dom_formosan_li_peirong_xu_weisheng_introduction_tru>": 259723,
+  "<dom_formosan_li_peizhen_wu_xingyu_shouhu_xing>": 259724,
+  "<dom_formosan_naomi_tsukida_correlative_clauses_seedi>": 259725,
+  "<dom_formosan_nowbucyang_truku_thesis>": 259726,
+  "<dom_formosan_old_texts>": 259727,
+  "<dom_formosan_paiwan_collart_zeitoun_time_reference>": 259728,
+  "<dom_formosan_paiwan_ho_five_dialects>": 259729,
+  "<dom_formosan_paiwanstories>": 259730,
+  "<dom_formosan_puyuma_chen_raising_to_object>": 259731,
+  "<dom_formosan_puyuma_katipol_kopfjagdriten>": 259732,
+  "<dom_formosan_puyuma_teng_pronominal_systems>": 259733,
+  "<dom_formosan_puyuma_teng_reference_grammar>": 259734,
+  "<dom_formosan_rik_bunun>": 259735,
+  "<dom_formosan_robert_blust_austronesian_homeland_ling>": 259736,
+  "<dom_formosan_rukai_mantauran_stories>": 259737,
+  "<dom_formosan_rukai_texts>": 259738,
+  "<dom_formosan_rukai_zeitoun_saying>": 259739,
+  "<dom_formosan_saaroa_pan_grammar>": 259740,
+  "<dom_formosan_sakizaya_affixes>": 259741,
+  "<dom_formosan_seals>": 259742,
+  "<dom_formosan_seediq_zhang_reference_grammar>": 259743,
+  "<dom_formosan_shigeru_tsuchida_kanakanavu_texts>": 259744,
+  "<dom_formosan_shigeru_tsuchida_reconstruction_proto_t>": 259745,
+  "<dom_formosan_shih_rukai_adverbial>": 259746,
+  "<dom_formosan_song_limei_introduction_kanakanavu_gram>": 259747,
+  "<dom_formosan_song_limei_introduction_seediq_grammar>": 259748,
+  "<dom_formosan_sung_kuo_descriptive_comparative_constr>": 259749,
+  "<dom_formosan_taoshan_elementary_school_atelier_hui_k>": 259750,
+  "<dom_formosan_tsukida_naomi_verb_classification_amis>": 259751,
+  "<dom_formosan_wei_matri_clan_lineage_system_ami>": 259752,
+  "<dom_formosan_wilang_yutas_videos>": 259753,
+  "<dom_formosan_wu_jinglan_introduction_amis_grammar>": 259754,
+  "<dom_formosan_yedda_palemeq_blog>": 259755,
+  "<dom_formosan_yeddas_blog>": 259756,
+  "<dom_formosan_yeh_meili_introduction_saisiyat_grammar>": 259757,
+  "<dom_formosan_zhang_yongli_pan_jiarong_introduction_t>": 259758,
+  "<dom_formosan_zheng_acl_2024>": 259759,
+  "<dom_formosan_zheng_data>": 259760,
+  "<dom_learning_vocab>": 259761,
+  "<dom_nine_level>": 259762,
+  "<dom_ntu>": 259763,
+  "<dom_picture_book>": 259764,
+  "<dom_picture_story>": 259765,
+  "<dom_presidential_apology>": 259766,
+  "<dom_reading_writing>": 259767,
+  "<dom_unknown>": 259768,
+  "<dom_youtube>": 259769,
+  "<src_ami>": 259770,
+  "<src_bnn>": 259771,
+  "<src_ckv>": 259772,
+  "<src_dru>": 259773,
+  "<src_eng>": 259774,
+  "<src_pwn>": 259775,
+  "<src_pyu>": 259776,
+  "<src_ssf>": 259777,
+  "<src_sxr>": 259778,
+  "<src_szy>": 259779,
+  "<src_tao>": 259780,
+  "<src_tay>": 259781,
+  "<src_trv>": 259782,
+  "<src_tsu>": 259783,
+  "<src_xnb>": 259784,
+  "<src_xsy>": 259785,
+  "<src_zh>": 259786,
+  "<to_ami>": 259787,
+  "<to_bnn>": 259788,
+  "<to_ckv>": 259789,
+  "<to_dru>": 259790,
+  "<to_eng>": 259791,
+  "<to_pwn>": 259792,
+  "<to_pyu>": 259793,
+  "<to_ssf>": 259794,
+  "<to_sxr>": 259795,
+  "<to_szy>": 259796,
+  "<to_tao>": 259797,
+  "<to_tay>": 259798,
+  "<to_trv>": 259799,
+  "<to_tsu>": 259800,
+  "<to_xnb>": 259801,
+  "<to_xsy>": 259802,
+  "<to_zh>": 259803,
   "ami_Latn": 259624,
   "bnn_Latn": 259625,
   "ckv_Latn": 259626,

config.json CHANGED Viewed

@@ -30,5 +30,5 @@
   "tokenizer_class": "NllbTokenizer",
   "transformers_version": "4.56.1",
   "use_cache": true,
-  "vocab_size": 259732
 }

   "tokenizer_class": "NllbTokenizer",
   "transformers_version": "4.56.1",
   "use_cache": true,
+  "vocab_size": 259804
 }

eval/metrics.json CHANGED Viewed

@@ -1,226 +1,511 @@
 {
-  "input": "/projects/prudlab/formosan_parallel_corpora/splits_zh_v1/big_corpus_zh_in_domain_hard.csv",
-  "model": "/scratch/scheppat/formosan_mt_experiments/runs/E3_zh_spm8192_f2zh_zh_e3_20260510-155741/final",
-  "tokenizer": "/scratch/scheppat/formosan_mt_experiments/runs/E3_zh_spm8192_f2zh_zh_e3_20260510-155741/final",
   "direction": "f2zh",
   "target_lang": "chinese",
   "bleu_tokenize": "zh",
   "split": "test",
-  "samples": 37435,
   "global": {
-    "samples": 37435,
-    "BLEU": 9.788873522245254,
-    "chrF2": 11.773185537762172,
-    "TER": 109.28997211806558
   },
   "by_language": {
     "ami": {
-      "samples": 4866,
-      "BLEU": 9.67664596964354,
-      "chrF2": 11.257236746158707,
-      "TER": 107.56081525312295
     },
     "bnn": {
-      "samples": 3223,
-      "BLEU": 9.438797365141522,
-      "chrF2": 10.522835474870561,
-      "TER": 105.0420168067227
     },
     "ckv": {
-      "samples": 1832,
-      "BLEU": 14.267338682634806,
-      "chrF2": 15.110294265244454,
-      "TER": 108.67237687366168
     },
     "dru": {
-      "samples": 3846,
-      "BLEU": 5.572741778754694,
-      "chrF2": 8.574754937863247,
-      "TER": 108.67220849247362
     },
     "pwn": {
-      "samples": 3221,
-      "BLEU": 7.858502138390441,
-      "chrF2": 9.953520893642228,
-      "TER": 106.32911392405062
     },
     "pyu": {
-      "samples": 2225,
-      "BLEU": 15.76590468147179,
-      "chrF2": 16.92931971633205,
-      "TER": 124.54545454545453
     },
     "ssf": {
-      "samples": 1180,
-      "BLEU": 14.731852581704961,
-      "chrF2": 16.35588870724647,
-      "TER": 106.68789808917198
     },
     "sxr": {
-      "samples": 1106,
-      "BLEU": 9.174525069515981,
-      "chrF2": 12.793965259986544,
-      "TER": 108.06591500433652
     },
     "szy": {
-      "samples": 1601,
-      "BLEU": 11.415668309044698,
-      "chrF2": 14.82761299370274,
-      "TER": 104.58435207823962
     },
     "tao": {
-      "samples": 1455,
-      "BLEU": 6.925709253385289,
-      "chrF2": 10.383676109942837,
-      "TER": 110.84583901773533
     },
     "tay": {
-      "samples": 4293,
-      "BLEU": 6.83184841971991,
-      "chrF2": 9.06833672004013,
-      "TER": 108.88795282224095
     },
     "trv": {
-      "samples": 4573,
-      "BLEU": 10.091903008657704,
-      "chrF2": 11.960338890715946,
-      "TER": 109.35416666666666
     },
     "tsu": {
-      "samples": 1250,
-      "BLEU": 8.564382999347023,
-      "chrF2": 11.415204943663671,
-      "TER": 126.83504340962905
     },
     "xnb": {
-      "samples": 1552,
-      "BLEU": 12.963985306907468,
-      "chrF2": 15.187610721437808,
-      "TER": 107.91100123609394
     },
     "xsy": {
-      "samples": 1212,
-      "BLEU": 14.445934092992472,
-      "chrF2": 16.31696596414603,
-      "TER": 106.79611650485437
     }
   },
   "by_source_bucket": {
-    "Formosan-100_Paiwan_Texts": {
-      "samples": 422,
-      "BLEU": 7.6358528438610955,
-      "chrF2": 8.786253362904043,
-      "TER": 101.16279069767442
     },
     "Formosan-Amis_myths_and_customs": {
-      "samples": 134,
-      "BLEU": 1.3834967492612475,
-      "chrF2": 3.592488073042097,
-      "TER": 104.22993492407808
     },
-    "Formosan-Old_Texts": {
-      "samples": 8,
-      "BLEU": 1.4198641212108538,
-      "chrF2": 4.022994803599383,
-      "TER": 100.0
     },
     "Formosan-PaiwanStories": {
-      "samples": 25,
-      "BLEU": 9.577235523415832,
-      "chrF2": 10.935197468097126,
-      "TER": 100.0
     },
-    "Formosan-Rik-Bunun": {
-      "samples": 225,
-      "BLEU": 1.6136362126845787,
-      "chrF2": 3.5441301624591186,
-      "TER": 104.88721804511279
     },
     "Formosan-ePark": {
-      "samples": 241,
-      "BLEU": 48.68162932289159,
-      "chrF2": 44.69693315027365,
-      "TER": 92.27642276422763
     },
     "Formosan-gitbook_translations": {
-      "samples": 93,
-      "BLEU": 3.9094128957776606,
-      "chrF2": 5.7929680228277745,
-      "TER": 100.0
     },
     "culture": {
-      "samples": 9919,
-      "BLEU": 6.549705149226013,
-      "chrF2": 8.025035060319183,
-      "TER": 106.47144948755492
     },
     "essays": {
-      "samples": 818,
-      "BLEU": 25.800187065494686,
-      "chrF2": 26.181381000355476,
-      "TER": 99.11209766925639
     },
     "ntu": {
-      "samples": 1950,
-      "BLEU": 10.931310727633916,
-      "chrF2": 11.96563503066236,
-      "TER": 102.64423076923077
     },
     "picture_book": {
-      "samples": 17068,
-      "BLEU": 11.985017998958577,
-      "chrF2": 14.739335041727964,
-      "TER": 112.57698727911128
     },
     "picture_story": {
-      "samples": 558,
-      "BLEU": 9.328673539386655,
-      "chrF2": 10.251406596288156,
-      "TER": 102.64900662251655
     },
     "presidential_apology": {
-      "samples": 8,
-      "BLEU": 3.2671889237498863,
-      "chrF2": 6.082422547344087,
-      "TER": 120.0
     },
     "reading_writing": {
-      "samples": 3711,
-      "BLEU": 7.548475476363713,
-      "chrF2": 8.953578848891137,
-      "TER": 109.46478873239435
     },
     "youtube": {
-      "samples": 2255,
-      "BLEU": 7.694036279346885,
-      "chrF2": 9.527966515275404,
-      "TER": 114.25576519916143
     }
   },
   "by_length_bin": {
     "004_008": {
-      "samples": 720,
-      "BLEU": 13.18000769800177,
-      "chrF2": 15.45653407120694,
-      "TER": 98.3225806451613
     },
     "009_016": {
-      "samples": 12886,
-      "BLEU": 11.737148849330296,
-      "chrF2": 13.767594349322392,
-      "TER": 106.43694493783305
     },
     "017_032": {
-      "samples": 19581,
-      "BLEU": 9.967727275396937,
-      "chrF2": 11.846840916327807,
-      "TER": 109.10390167214521
     },
     "033_plus": {
-      "samples": 4248,
-      "BLEU": 7.733152016372462,
-      "chrF2": 10.009223680215971,
-      "TER": 118.41055006073226
     }
   }
 }

 {
+  "input": "/projects/prudlab/formosan_parallel_corpora/private_no_bible/big_corpus_zh_in_domain_hard.csv",
+  "model": "/scratch/scheppat/projects/mt/formosan_mt_experiments/runs/private_no_bible/v1_spm8192_f2zh_20260712-232900/best",
+  "tokenizer": "/scratch/scheppat/projects/mt/formosan_mt_experiments/runs/private_no_bible/v1_spm8192_f2zh_20260712-232900/best",
   "direction": "f2zh",
   "target_lang": "chinese",
   "bleu_tokenize": "zh",
   "split": "test",
+  "samples": 61384,
   "global": {
+    "samples": 61384,
+    "BLEU": 10.576691446569214,
+    "chrF2": 12.391218687865106,
+    "TER": 115.75207512366899,
+    "exact_match_rate": 0.01397758373517529,
+    "empty_output_rate": 0.0,
+    "character_length_ratio": 0.6925750688613788
   },
   "by_language": {
     "ami": {
+      "samples": 10929,
+      "BLEU": 7.907158388525328,
+      "chrF2": 10.070412232114371,
+      "TER": 115.26327734906945,
+      "exact_match_rate": 0.010613962851130022,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.638478344685773
     },
     "bnn": {
+      "samples": 5450,
+      "BLEU": 9.239755649716296,
+      "chrF2": 10.905172805141694,
+      "TER": 120.58014216922723,
+      "exact_match_rate": 0.011376146788990826,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.6297844327645479
     },
     "ckv": {
+      "samples": 2852,
+      "BLEU": 15.762605478682843,
+      "chrF2": 16.76866562299388,
+      "TER": 108.04521276595744,
+      "exact_match_rate": 0.011921458625525946,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.7161089052750993
     },
     "dru": {
+      "samples": 5524,
+      "BLEU": 8.136977886054543,
+      "chrF2": 10.050861113586718,
+      "TER": 127.32970671712394,
+      "exact_match_rate": 0.014301230992034758,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.6891913918113896
     },
     "pwn": {
+      "samples": 5519,
+      "BLEU": 8.80996385561619,
+      "chrF2": 10.986609677668069,
+      "TER": 111.14936146585231,
+      "exact_match_rate": 0.010871534698314912,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.6481772904433953
     },
     "pyu": {
+      "samples": 4105,
+      "BLEU": 13.101948522565348,
+      "chrF2": 15.116067674782407,
+      "TER": 107.76247540690396,
+      "exact_match_rate": 0.01900121802679659,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.680589070537938
     },
     "ssf": {
+      "samples": 1670,
+      "BLEU": 14.866297961190686,
+      "chrF2": 16.293901223953522,
+      "TER": 110.55618615209988,
+      "exact_match_rate": 0.015568862275449102,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.724459557930532
     },
     "sxr": {
+      "samples": 1539,
+      "BLEU": 8.570268882776618,
+      "chrF2": 11.04003820559944,
+      "TER": 109.40737670949025,
+      "exact_match_rate": 0.010396361273554255,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.715325101193831
     },
     "szy": {
+      "samples": 2147,
+      "BLEU": 12.583295396175457,
+      "chrF2": 15.787293357235924,
+      "TER": 110.10262989095574,
+      "exact_match_rate": 0.021891010712622262,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.6639051190372373
     },
     "tao": {
+      "samples": 1802,
+      "BLEU": 8.318439478906354,
+      "chrF2": 11.299683524234275,
+      "TER": 109.37062937062937,
+      "exact_match_rate": 0.009988901220865706,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.6603234122334193
     },
     "tay": {
+      "samples": 6395,
+      "BLEU": 8.915637694044362,
+      "chrF2": 10.23849545512742,
+      "TER": 125.91745218266679,
+      "exact_match_rate": 0.012197028928850665,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.7223793403290757
     },
     "trv": {
+      "samples": 6689,
+      "BLEU": 17.65835020427611,
+      "chrF2": 18.00769858472333,
+      "TER": 107.97406438402912,
+      "exact_match_rate": 0.023620870085214532,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.881449702993361
     },
     "tsu": {
+      "samples": 1889,
+      "BLEU": 8.004987496560164,
+      "chrF2": 10.402536265430221,
+      "TER": 115.47044632086852,
+      "exact_match_rate": 0.009528851244044468,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.6691029709510202
     },
     "xnb": {
+      "samples": 2927,
+      "BLEU": 17.479049560336957,
+      "chrF2": 18.514675122197012,
+      "TER": 111.06577052328373,
+      "exact_match_rate": 0.01400751622822002,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.7461520246270423
     },
     "xsy": {
+      "samples": 1947,
+      "BLEU": 9.938882396702315,
+      "chrF2": 12.24802454126081,
+      "TER": 125.58068459657702,
+      "exact_match_rate": 0.01386748844375963,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.7180976234053972
     }
   },
   "by_source_bucket": {
+    "Formosan-Academia-Sinica-Oral-Legends": {
+      "samples": 4,
+      "BLEU": 13.708780144263425,
+      "chrF2": 15.041820333450085,
+      "TER": 100.0,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.6444444444444445
+    },
+    "Formosan-Amis-Lifok-Dongi-Saopo-Kimakimad": {
+      "samples": 1604,
+      "BLEU": 2.3379152308621682,
+      "chrF2": 5.912072736378735,
+      "TER": 146.6566113624937,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.6363730379426839
     },
     "Formosan-Amis_myths_and_customs": {
+      "samples": 7,
+      "BLEU": 1.6263246843084138,
+      "chrF2": 4.5570794176045215,
+      "TER": 100.0,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.36831604150039904
+    },
+    "Formosan-CIP-Atayal-Grammar-Overview": {
+      "samples": 56,
+      "BLEU": 17.073688928053834,
+      "chrF2": 18.201008148785526,
+      "TER": 93.24324324324324,
+      "exact_match_rate": 0.07142857142857142,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.8041692987997473
+    },
+    "Formosan-Glosbe": {
+      "samples": 21,
+      "BLEU": 3.3978952817507193,
+      "chrF2": 6.608750008028403,
+      "TER": 100.0,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.8833333333333333
+    },
+    "Formosan-Hala-Saku-La-Videos": {
+      "samples": 24,
+      "BLEU": 3.871055448346106,
+      "chrF2": 9.345209439285906,
+      "TER": 108.8235294117647,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.5491445823549145
+    },
+    "Formosan-Huang-Mei-Chin-Atayal-Reference-Grammar": {
+      "samples": 2,
+      "BLEU": 15.154686451352852,
+      "chrF2": 30.96466834342189,
+      "TER": 100.0,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.6216216216216216
+    },
+    "Formosan-ILRDF-Tousvusvutu-Kita": {
+      "samples": 738,
+      "BLEU": 4.529526769930424,
+      "chrF2": 6.126303744736681,
+      "TER": 122.51297090006767,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.6800853427800814
+    },
+    "Formosan-Indigenous-Language-Literary-Awards": {
+      "samples": 3206,
+      "BLEU": 1.7454225871296085,
+      "chrF2": 3.2831850026941756,
+      "TER": 177.2076961026147,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.7565487244648617
+    },
+    "Formosan-Kavalan-Zhang-Reference-Grammar": {
+      "samples": 2,
+      "BLEU": 13.555499052342826,
+      "chrF2": 12.73179069019462,
+      "TER": 87.09677419354838,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.9154929577464789
+    },
+    "Formosan-Li-Peirong-Xu-Weisheng-Introduction-Truku-Grammar": {
+      "samples": 122,
+      "BLEU": 23.717942859641965,
+      "chrF2": 28.708599449512906,
+      "TER": 94.44444444444444,
+      "exact_match_rate": 0.07377049180327869,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.863456090651558
+    },
+    "Formosan-Li-Peizhen-Wu-Xingyu-Shouhu-Xing": {
+      "samples": 6,
+      "BLEU": 1.9271815425189616,
+      "chrF2": 5.233801820077148,
+      "TER": 100.0,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.5118483412322274
     },
+    "Formosan-Nowbucyang-Truku-Thesis": {
+      "samples": 64,
+      "BLEU": 33.03227143741538,
+      "chrF2": 35.4877874425264,
+      "TER": 81.69014084507043,
+      "exact_match_rate": 0.109375,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.803030303030303
     },
     "Formosan-PaiwanStories": {
+      "samples": 18,
+      "BLEU": 8.869948503140135,
+      "chrF2": 10.358317911576524,
+      "TER": 109.09090909090908,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.7548262548262549
     },
+    "Formosan-SEALS": {
+      "samples": 16,
+      "BLEU": 5.401492843871145,
+      "chrF2": 6.2423672065754525,
+      "TER": 175.0,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.814629258517034
+    },
+    "Formosan-Seediq-Zhang-Reference-Grammar": {
+      "samples": 15,
+      "BLEU": 10.731865242690281,
+      "chrF2": 17.52649006585302,
+      "TER": 88.23529411764706,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.7033898305084746
+    },
+    "Formosan-Song-Limei-Introduction-Kanakanavu-Grammar": {
+      "samples": 104,
+      "BLEU": 21.222612310827056,
+      "chrF2": 23.44610806619767,
+      "TER": 103.84615384615385,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.8052876027152555
+    },
+    "Formosan-Song-Limei-Introduction-Seediq-Grammar": {
+      "samples": 107,
+      "BLEU": 28.635676865278743,
+      "chrF2": 31.56948663323015,
+      "TER": 95.72649572649573,
+      "exact_match_rate": 0.08411214953271028,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.8722649319929036
+    },
+    "Formosan-Wilang-Yutas-Videos": {
+      "samples": 10,
+      "BLEU": 9.324906605075657,
+      "chrF2": 11.029154358634294,
+      "TER": 100.0,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.7066666666666667
+    },
+    "Formosan-Wu-Jinglan-Introduction-Amis-Grammar": {
+      "samples": 34,
+      "BLEU": 14.528023877356814,
+      "chrF2": 23.12502283686413,
+      "TER": 102.27272727272727,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.5162019593067069
+    },
+    "Formosan-Yeh-Meili-Introduction-Saisiyat-Grammar": {
+      "samples": 4,
+      "BLEU": 3.851546244568978,
+      "chrF2": 6.389276621044812,
+      "TER": 100.0,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.3870967741935484
+    },
+    "Formosan-Zhang-Yongli-Pan-Jiarong-Introduction-Tsou-Grammar": {
+      "samples": 37,
+      "BLEU": 6.388565113016769,
+      "chrF2": 9.86522307741866,
+      "TER": 100.0,
+      "exact_match_rate": 0.05405405405405406,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.6717557251908397
+    },
+    "Formosan-Zheng-ACL-2024": {
+      "samples": 4947,
+      "BLEU": 20.15707801656887,
+      "chrF2": 20.056972103148482,
+      "TER": 101.21051636088518,
+      "exact_match_rate": 0.003840711542348898,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.7978903211354796
+    },
+    "Formosan-amis_tung-chiou": {
+      "samples": 4,
+      "BLEU": 2.270450618468058,
+      "chrF2": 4.86818640558682,
+      "TER": 102.94117647058823,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.8408941485864563
     },
     "Formosan-ePark": {
+      "samples": 5491,
+      "BLEU": 36.11130715740376,
+      "chrF2": 32.81108828189628,
+      "TER": 94.7163014109877,
+      "exact_match_rate": 0.07412128938262612,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.7846655150753769
     },
     "Formosan-gitbook_translations": {
+      "samples": 62,
+      "BLEU": 5.336287344219993,
+      "chrF2": 11.250725678537165,
+      "TER": 97.20496894409938,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.59533527696793
     },
     "culture": {
+      "samples": 11335,
+      "BLEU": 4.230999106074682,
+      "chrF2": 6.803049833850721,
+      "TER": 103.05977030765361,
+      "exact_match_rate": 8.82223202470225e-05,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.5262328026781123
     },
     "essays": {
+      "samples": 3176,
+      "BLEU": 16.22959311401136,
+      "chrF2": 16.18942996487404,
+      "TER": 117.39726027397259,
+      "exact_match_rate": 0.010390428211586901,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.7892565422616429
+    },
+    "nine_level": {
+      "samples": 1933,
+      "BLEU": 38.94085170738614,
+      "chrF2": 35.54484444458279,
+      "TER": 91.82939362795479,
+      "exact_match_rate": 0.08277289187790998,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.8508923757909372
     },
     "ntu": {
+      "samples": 2884,
+      "BLEU": 13.473343908627347,
+      "chrF2": 14.92087756037663,
+      "TER": 108.14696485623003,
+      "exact_match_rate": 0.0038141470180305132,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.7321011104617183
     },
     "picture_book": {
+      "samples": 13023,
+      "BLEU": 13.78815631073597,
+      "chrF2": 16.69686326560025,
+      "TER": 112.86555398764051,
+      "exact_match_rate": 0.014359210627351609,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.7865734687720004
     },
     "picture_story": {
+      "samples": 729,
+      "BLEU": 6.502675604326252,
+      "chrF2": 8.737286139710305,
+      "TER": 109.9591836734694,
+      "exact_match_rate": 0.0013717421124828531,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.6134348528296911
     },
     "presidential_apology": {
+      "samples": 482,
+      "BLEU": 3.4850550479787836,
+      "chrF2": 5.319974289887845,
+      "TER": 108.20930232558139,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.5924246272176469
     },
     "reading_writing": {
+      "samples": 4793,
+      "BLEU": 5.319686455411381,
+      "chrF2": 7.422908022519387,
+      "TER": 150.53610644619556,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.6066766246521526
     },
     "youtube": {
+      "samples": 6324,
+      "BLEU": 7.76714599062836,
+      "chrF2": 9.71030705405905,
+      "TER": 113.357928884931,
+      "exact_match_rate": 0.001265022137887413,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.6960417107375285
     }
   },
   "by_length_bin": {
     "004_008": {
+      "samples": 1326,
+      "BLEU": 33.75734846323192,
+      "chrF2": 29.923056248729267,
+      "TER": 86.28858578607323,
+      "exact_match_rate": 0.14630467571644043,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.8196241382566815
     },
     "009_016": {
+      "samples": 16531,
+      "BLEU": 18.47308209896322,
+      "chrF2": 18.70713057005937,
+      "TER": 100.480836562252,
+      "exact_match_rate": 0.029096848345532635,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.7163221383373605
     },
     "017_032": {
+      "samples": 34591,
+      "BLEU": 11.382747289801152,
+      "chrF2": 13.351430361108669,
+      "TER": 113.17818330941152,
+      "exact_match_rate": 0.005290393454944928,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.6985298702060786
     },
     "033_plus": {
+      "samples": 8936,
+      "BLEU": 4.7159182063106675,
+      "chrF2": 7.239179763126493,
+      "TER": 136.46777442094663,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.6668947584508006
     }
   }
 }

experiment_metadata.json CHANGED Viewed

@@ -1,4 +1,183 @@
 {
-  "step": 300000,
-  "direction": "f2zh"
 }

 {
+  "step": 270000,
+  "best_metric": "chrF2",
+  "best_value": 22.942873146989022,
+  "direction": "f2zh",
+  "validation": {
+    "mean_token_loss": 2.420696591592408,
+    "ppl": 11.253695815033172,
+    "by_language": {
+      "ami": 2.5921270985999834,
+      "bnn": 2.1358773591268245,
+      "ckv": 1.9623805818177094,
+      "dru": 2.014338453068967,
+      "pwn": 2.489123042752967,
+      "pyu": 1.8734191935536901,
+      "ssf": 2.1260528645607564,
+      "sxr": 2.578650267016047,
+      "szy": 2.6909024219707973,
+      "tao": 2.8988929454858394,
+      "tay": 1.8934121017405425,
+      "trv": 3.2390189123141764,
+      "tsu": 2.578946331391228,
+      "xnb": 1.8004895268200158,
+      "xsy": 2.6114685328135936
+    },
+    "generation": {
+      "samples": 1920,
+      "bleu_tokenize": "zh",
+      "global": {
+        "BLEU": 23.57852524101978,
+        "chrF2": 22.942873146989022,
+        "TER": 103.9412997903564,
+        "exact_match_rate": 0.020833333333333332,
+        "empty_output_rate": 0.0,
+        "character_length_ratio": 0.7932179967578198
+      },
+      "by_language": {
+        "ami": {
+          "samples": 128,
+          "BLEU": 19.55644041328223,
+          "chrF2": 18.577728465895298,
+          "TER": 104.81283422459893,
+          "exact_match_rate": 0.0078125,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.7593659942363112
+        },
+        "bnn": {
+          "samples": 128,
+          "BLEU": 25.380621452125194,
+          "chrF2": 23.79299268084961,
+          "TER": 103.28947368421053,
+          "exact_match_rate": 0.0078125,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.7781674704077159
+        },
+        "ckv": {
+          "samples": 128,
+          "BLEU": 27.113629785341512,
+          "chrF2": 25.146757668610668,
+          "TER": 103.82165605095541,
+          "exact_match_rate": 0.015625,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.8045977011494253
+        },
+        "dru": {
+          "samples": 128,
+          "BLEU": 26.22655487827295,
+          "chrF2": 25.694007229053373,
+          "TER": 105.22875816993465,
+          "exact_match_rate": 0.0,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.7707253886010362
+        },
+        "pwn": {
+          "samples": 128,
+          "BLEU": 21.141247720936498,
+          "chrF2": 19.70542420778234,
+          "TER": 101.65745856353593,
+          "exact_match_rate": 0.0078125,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.7858619264158744
+        },
+        "pyu": {
+          "samples": 128,
+          "BLEU": 28.010020387816798,
+          "chrF2": 26.26557564859005,
+          "TER": 113.88888888888889,
+          "exact_match_rate": 0.015625,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.819854514334617
+        },
+        "ssf": {
+          "samples": 128,
+          "BLEU": 28.980899971001712,
+          "chrF2": 27.362664735428773,
+          "TER": 99.32432432432432,
+          "exact_match_rate": 0.0234375,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.8144778481012658
+        },
+        "sxr": {
+          "samples": 128,
+          "BLEU": 21.355107092623804,
+          "chrF2": 22.270553009964367,
+          "TER": 97.97297297297297,
+          "exact_match_rate": 0.015625,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.793171114599686
+        },
+        "szy": {
+          "samples": 128,
+          "BLEU": 21.52598808800314,
+          "chrF2": 21.39925099180053,
+          "TER": 98.66666666666667,
+          "exact_match_rate": 0.03125,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.7829099307159353
+        },
+        "tao": {
+          "samples": 128,
+          "BLEU": 22.207667963076098,
+          "chrF2": 23.056824267735124,
+          "TER": 109.39597315436242,
+          "exact_match_rate": 0.03125,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.8163511880789368
+        },
+        "tay": {
+          "samples": 128,
+          "BLEU": 28.009733813829975,
+          "chrF2": 26.444824918215147,
+          "TER": 102.29885057471265,
+          "exact_match_rate": 0.03125,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.7627254509018037
+        },
+        "trv": {
+          "samples": 128,
+          "BLEU": 13.93836812194567,
+          "chrF2": 14.344449649929345,
+          "TER": 125.88235294117646,
+          "exact_match_rate": 0.015625,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.7944753659039373
+        },
+        "tsu": {
+          "samples": 128,
+          "BLEU": 24.78776703981478,
+          "chrF2": 23.942995364265407,
+          "TER": 95.625,
+          "exact_match_rate": 0.03125,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.7733105218135158
+        },
+        "xnb": {
+          "samples": 128,
+          "BLEU": 33.44577807814948,
+          "chrF2": 32.87463813651187,
+          "TER": 92.90322580645162,
+          "exact_match_rate": 0.0625,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.8204047217537943
+        },
+        "xsy": {
+          "samples": 128,
+          "BLEU": 22.10377012604304,
+          "chrF2": 22.747005676693703,
+          "TER": 103.18471337579618,
+          "exact_match_rate": 0.015625,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.8155631986242476
+        }
+      }
+    },
+    "step": 270000,
+    "selection": {
+      "metric": "chrF2",
+      "value": 22.942873146989022,
+      "improved": true,
+      "best_value_before_eval": 22.794952357181167
+    }
+  }
 }

model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:30756965a68b24356bd940e91aed60a0d38f9866899caf4caf61e1e12b3aae46
-size 2474797408

 version https://git-lfs.github.com/spec/v1
+oid sha256:14395539c53293e66e69aef90ba1247992b9e5b89a083890d7cae56873a9a0e7
+size 2475092320

sentencepiece.bpe.model CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5450fbcd0f7a01bc2a8d6f39edcb102752660081bfd686b9e46d76a399a31b3f
-size 4900258

 version https://git-lfs.github.com/spec/v1
+oid sha256:0571a653d72d4d5a892f302084363ae86de93f9da19521a9bd3498b60819b960
+size 4898297

special_tokens_map.json CHANGED Viewed

@@ -231,17 +231,24 @@
     "<dialect_hengchun>",
     "<dialect_jianhe>",
     "<dialect_junqun>",
     "<dialect_kaqun>",
     "<dialect_luanqun>",
     "<dialect_malan>",
     "<dialect_maolin>",
     "<dialect_nanwang>",
     "<dialect_northern>",
     "<dialect_sekolik>",
     "<dialect_southern>",
     "<dialect_tanqun>",
     "<dialect_tegudaya>",
     "<dialect_truku>",
     "<dialect_unknown>",
     "<dialect_wanda>",
     "<dialect_wanshan>",
@@ -249,7 +256,8 @@
     "<dialect_wutai>",
     "<dialect_xiqun>",
     "<dialect_xiuguluan>",
-    "<dialect_yilan_zeaol>",
     "<dialect_zeaol>",
     "<dialect_zhiben>",
     "<dialect_zhuoqun>",
@@ -257,16 +265,80 @@
     "<dom_culture>",
     "<dom_dictionary>",
     "<dom_essays>",
     "<dom_formosan_100_paiwan_texts>",
     "<dom_formosan_amis_myths_and_customs>",
     "<dom_formosan_epark>",
     "<dom_formosan_gitbook_translations>",
     "<dom_formosan_old_texts>",
     "<dom_formosan_paiwanstories>",
     "<dom_formosan_rik_bunun>",
     "<dom_formosan_seals>",
     "<dom_formosan_wilang_yutas_videos>",
     "<dom_formosan_yeddas_blog>",
     "<dom_formosan_zheng_data>",
     "<dom_learning_vocab>",
     "<dom_nine_level>",

     "<dialect_hengchun>",
     "<dialect_jianhe>",
     "<dialect_junqun>",
+    "<dialect_kanakanavu>",
     "<dialect_kaqun>",
+    "<dialect_kavalan>",
     "<dialect_luanqun>",
     "<dialect_malan>",
     "<dialect_maolin>",
     "<dialect_nanwang>",
     "<dialect_northern>",
+    "<dialect_saaroa>",
+    "<dialect_saisiyat>",
+    "<dialect_sakizaya>",
     "<dialect_sekolik>",
     "<dialect_southern>",
     "<dialect_tanqun>",
     "<dialect_tegudaya>",
+    "<dialect_thao>",
     "<dialect_truku>",
+    "<dialect_tsou>",
     "<dialect_unknown>",
     "<dialect_wanda>",
     "<dialect_wanshan>",
     "<dialect_wutai>",
     "<dialect_xiqun>",
     "<dialect_xiuguluan>",
+    "<dialect_yami>",
+    "<dialect_yilanzeaol>",
     "<dialect_zeaol>",
     "<dialect_zhiben>",
     "<dialect_zhuoqun>",
     "<dom_culture>",
     "<dom_dictionary>",
     "<dom_essays>",
+    "<dom_ethnography_journal_ritual_song>",
     "<dom_formosan_100_paiwan_texts>",
+    "<dom_formosan_academia_sinica_oral_legends>",
+    "<dom_formosan_amis_adversative_constructions>",
+    "<dom_formosan_amis_kavalan_lin_interrogative_verbs>",
+    "<dom_formosan_amis_kuo_sung_comparative_constructions>",
+    "<dom_formosan_amis_lifok_dongi_saopo_kimakimad>",
     "<dom_formosan_amis_myths_and_customs>",
+    "<dom_formosan_amis_pa_verbs>",
+    "<dom_formosan_amis_serial_verb_constructions>",
+    "<dom_formosan_amis_tung_chiou>",
+    "<dom_formosan_asai_sedik_language>",
+    "<dom_formosan_bunun_debusser_dissertation>",
+    "<dom_formosan_bunun_moriguchi_northern_texts>",
+    "<dom_formosan_bunun_topic_focus>",
+    "<dom_formosan_chen_fukuda_relabeling_ergative>",
+    "<dom_formosan_chen_fukuda_three_ways_steal>",
+    "<dom_formosan_cip_atayal_grammar_overview>",
+    "<dom_formosan_dean_johnson_year_clouds_smangus>",
+    "<dom_formosan_dorinda_tsai_hsiu_liu_neutral_imperfect>",
+    "<dom_formosan_egerod_origin_headhunting_atayal_text_v>",
+    "<dom_formosan_egerod_soren_word_order_word_classes_at>",
+    "<dom_formosan_elizabeth_zeitoun_chen_huei_wu_overview>",
     "<dom_formosan_epark>",
+    "<dom_formosan_ferrell_taiwan_aboriginal_groups_proble>",
     "<dom_formosan_gitbook_translations>",
+    "<dom_formosan_glosbe>",
+    "<dom_formosan_hala_saku_la_videos>",
+    "<dom_formosan_holmer_seediq>",
+    "<dom_formosan_huang_grammaticalization_squliq_atayal>",
+    "<dom_formosan_huang_mei_chin_atayal_reference_grammar>",
+    "<dom_formosan_huteson_rukai_survey>",
+    "<dom_formosan_ilrdf_42_language_practice_word_lists>",
+    "<dom_formosan_ilrdf_tousvusvutu_kita>",
+    "<dom_formosan_indigenous_language_literary_awards>",
+    "<dom_formosan_kanakanavu_texts>",
+    "<dom_formosan_kavalan_zhang_reference_grammar>",
+    "<dom_formosan_li_peirong_xu_weisheng_introduction_tru>",
+    "<dom_formosan_li_peizhen_wu_xingyu_shouhu_xing>",
+    "<dom_formosan_naomi_tsukida_correlative_clauses_seedi>",
+    "<dom_formosan_nowbucyang_truku_thesis>",
     "<dom_formosan_old_texts>",
+    "<dom_formosan_paiwan_collart_zeitoun_time_reference>",
+    "<dom_formosan_paiwan_ho_five_dialects>",
     "<dom_formosan_paiwanstories>",
+    "<dom_formosan_puyuma_chen_raising_to_object>",
+    "<dom_formosan_puyuma_katipol_kopfjagdriten>",
+    "<dom_formosan_puyuma_teng_pronominal_systems>",
+    "<dom_formosan_puyuma_teng_reference_grammar>",
     "<dom_formosan_rik_bunun>",
+    "<dom_formosan_robert_blust_austronesian_homeland_ling>",
+    "<dom_formosan_rukai_mantauran_stories>",
+    "<dom_formosan_rukai_texts>",
+    "<dom_formosan_rukai_zeitoun_saying>",
+    "<dom_formosan_saaroa_pan_grammar>",
+    "<dom_formosan_sakizaya_affixes>",
     "<dom_formosan_seals>",
+    "<dom_formosan_seediq_zhang_reference_grammar>",
+    "<dom_formosan_shigeru_tsuchida_kanakanavu_texts>",
+    "<dom_formosan_shigeru_tsuchida_reconstruction_proto_t>",
+    "<dom_formosan_shih_rukai_adverbial>",
+    "<dom_formosan_song_limei_introduction_kanakanavu_gram>",
+    "<dom_formosan_song_limei_introduction_seediq_grammar>",
+    "<dom_formosan_sung_kuo_descriptive_comparative_constr>",
+    "<dom_formosan_taoshan_elementary_school_atelier_hui_k>",
+    "<dom_formosan_tsukida_naomi_verb_classification_amis>",
+    "<dom_formosan_wei_matri_clan_lineage_system_ami>",
     "<dom_formosan_wilang_yutas_videos>",
+    "<dom_formosan_wu_jinglan_introduction_amis_grammar>",
+    "<dom_formosan_yedda_palemeq_blog>",
     "<dom_formosan_yeddas_blog>",
+    "<dom_formosan_yeh_meili_introduction_saisiyat_grammar>",
+    "<dom_formosan_zhang_yongli_pan_jiarong_introduction_t>",
+    "<dom_formosan_zheng_acl_2024>",
     "<dom_formosan_zheng_data>",
     "<dom_learning_vocab>",
     "<dom_nine_level>",

tokenizer_config.json CHANGED Viewed

@@ -1881,7 +1881,7 @@
       "special": true
     },
     "259652": {
-      "content": "<dialect_kaqun>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1889,7 +1889,7 @@
       "special": true
     },
     "259653": {
-      "content": "<dialect_luanqun>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1897,7 +1897,7 @@
       "special": true
     },
     "259654": {
-      "content": "<dialect_malan>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1905,7 +1905,7 @@
       "special": true
     },
     "259655": {
-      "content": "<dialect_maolin>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1913,7 +1913,7 @@
       "special": true
     },
     "259656": {
-      "content": "<dialect_nanwang>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1921,7 +1921,7 @@
       "special": true
     },
     "259657": {
-      "content": "<dialect_northern>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1929,7 +1929,7 @@
       "special": true
     },
     "259658": {
-      "content": "<dialect_sekolik>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1937,7 +1937,7 @@
       "special": true
     },
     "259659": {
-      "content": "<dialect_southern>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1945,7 +1945,7 @@
       "special": true
     },
     "259660": {
-      "content": "<dialect_tanqun>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1953,7 +1953,7 @@
       "special": true
     },
     "259661": {
-      "content": "<dialect_tegudaya>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1961,7 +1961,7 @@
       "special": true
     },
     "259662": {
-      "content": "<dialect_truku>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1969,7 +1969,7 @@
       "special": true
     },
     "259663": {
-      "content": "<dialect_unknown>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1977,7 +1977,7 @@
       "special": true
     },
     "259664": {
-      "content": "<dialect_wanda>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1985,7 +1985,7 @@
       "special": true
     },
     "259665": {
-      "content": "<dialect_wanshan>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1993,7 +1993,7 @@
       "special": true
     },
     "259666": {
-      "content": "<dialect_wenshui>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2001,7 +2001,7 @@
       "special": true
     },
     "259667": {
-      "content": "<dialect_wutai>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2009,7 +2009,7 @@
       "special": true
     },
     "259668": {
-      "content": "<dialect_xiqun>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2017,7 +2017,7 @@
       "special": true
     },
     "259669": {
-      "content": "<dialect_xiuguluan>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2025,7 +2025,7 @@
       "special": true
     },
     "259670": {
-      "content": "<dialect_yilan_zeaol>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2033,7 +2033,7 @@
       "special": true
     },
     "259671": {
-      "content": "<dialect_zeaol>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2041,7 +2041,7 @@
       "special": true
     },
     "259672": {
-      "content": "<dialect_zhiben>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2049,7 +2049,7 @@
       "special": true
     },
     "259673": {
-      "content": "<dialect_zhuoqun>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2057,7 +2057,7 @@
       "special": true
     },
     "259674": {
-      "content": "<dom_classroom_context>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2065,7 +2065,7 @@
       "special": true
     },
     "259675": {
-      "content": "<dom_culture>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2073,7 +2073,7 @@
       "special": true
     },
     "259676": {
-      "content": "<dom_dictionary>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2081,7 +2081,7 @@
       "special": true
     },
     "259677": {
-      "content": "<dom_essays>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2089,7 +2089,7 @@
       "special": true
     },
     "259678": {
-      "content": "<dom_formosan_100_paiwan_texts>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2097,7 +2097,7 @@
       "special": true
     },
     "259679": {
-      "content": "<dom_formosan_amis_myths_and_customs>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2105,7 +2105,7 @@
       "special": true
     },
     "259680": {
-      "content": "<dom_formosan_epark>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2113,7 +2113,7 @@
       "special": true
     },
     "259681": {
-      "content": "<dom_formosan_gitbook_translations>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2121,7 +2121,7 @@
       "special": true
     },
     "259682": {
-      "content": "<dom_formosan_old_texts>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2129,7 +2129,7 @@
       "special": true
     },
     "259683": {
-      "content": "<dom_formosan_paiwanstories>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2137,7 +2137,7 @@
       "special": true
     },
     "259684": {
-      "content": "<dom_formosan_rik_bunun>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2145,7 +2145,7 @@
       "special": true
     },
     "259685": {
-      "content": "<dom_formosan_seals>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2153,7 +2153,7 @@
       "special": true
     },
     "259686": {
-      "content": "<dom_formosan_wilang_yutas_videos>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2161,7 +2161,7 @@
       "special": true
     },
     "259687": {
-      "content": "<dom_formosan_yeddas_blog>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2169,7 +2169,7 @@
       "special": true
     },
     "259688": {
-      "content": "<dom_formosan_zheng_data>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2177,7 +2177,7 @@
       "special": true
     },
     "259689": {
-      "content": "<dom_learning_vocab>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2185,7 +2185,7 @@
       "special": true
     },
     "259690": {
-      "content": "<dom_nine_level>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2193,7 +2193,7 @@
       "special": true
     },
     "259691": {
-      "content": "<dom_ntu>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2201,7 +2201,7 @@
       "special": true
     },
     "259692": {
-      "content": "<dom_picture_book>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2209,7 +2209,7 @@
       "special": true
     },
     "259693": {
-      "content": "<dom_picture_story>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2217,7 +2217,7 @@
       "special": true
     },
     "259694": {
-      "content": "<dom_presidential_apology>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2225,7 +2225,7 @@
       "special": true
     },
     "259695": {
-      "content": "<dom_reading_writing>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2233,7 +2233,7 @@
       "special": true
     },
     "259696": {
-      "content": "<dom_unknown>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2241,7 +2241,7 @@
       "special": true
     },
     "259697": {
-      "content": "<dom_youtube>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2249,7 +2249,7 @@
       "special": true
     },
     "259698": {
-      "content": "<src_ami>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2257,7 +2257,7 @@
       "special": true
     },
     "259699": {
-      "content": "<src_bnn>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2265,7 +2265,7 @@
       "special": true
     },
     "259700": {
-      "content": "<src_ckv>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2273,7 +2273,7 @@
       "special": true
     },
     "259701": {
-      "content": "<src_dru>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2281,7 +2281,7 @@
       "special": true
     },
     "259702": {
-      "content": "<src_eng>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2289,7 +2289,7 @@
       "special": true
     },
     "259703": {
-      "content": "<src_pwn>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2297,7 +2297,7 @@
       "special": true
     },
     "259704": {
-      "content": "<src_pyu>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2305,7 +2305,7 @@
       "special": true
     },
     "259705": {
-      "content": "<src_ssf>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2313,7 +2313,7 @@
       "special": true
     },
     "259706": {
-      "content": "<src_sxr>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2321,7 +2321,7 @@
       "special": true
     },
     "259707": {
-      "content": "<src_szy>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2329,7 +2329,7 @@
       "special": true
     },
     "259708": {
-      "content": "<src_tao>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2337,7 +2337,7 @@
       "special": true
     },
     "259709": {
-      "content": "<src_tay>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2345,7 +2345,7 @@
       "special": true
     },
     "259710": {
-      "content": "<src_trv>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2353,7 +2353,7 @@
       "special": true
     },
     "259711": {
-      "content": "<src_tsu>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2361,7 +2361,7 @@
       "special": true
     },
     "259712": {
-      "content": "<src_xnb>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2369,7 +2369,7 @@
       "special": true
     },
     "259713": {
-      "content": "<src_xsy>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2377,7 +2377,7 @@
       "special": true
     },
     "259714": {
-      "content": "<src_zh>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2385,7 +2385,7 @@
       "special": true
     },
     "259715": {
-      "content": "<to_ami>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2393,7 +2393,7 @@
       "special": true
     },
     "259716": {
-      "content": "<to_bnn>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2401,7 +2401,7 @@
       "special": true
     },
     "259717": {
-      "content": "<to_ckv>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2409,7 +2409,7 @@
       "special": true
     },
     "259718": {
-      "content": "<to_dru>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2417,7 +2417,7 @@
       "special": true
     },
     "259719": {
-      "content": "<to_eng>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2425,7 +2425,7 @@
       "special": true
     },
     "259720": {
-      "content": "<to_pwn>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2433,7 +2433,7 @@
       "special": true
     },
     "259721": {
-      "content": "<to_pyu>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2441,7 +2441,7 @@
       "special": true
     },
     "259722": {
-      "content": "<to_ssf>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2449,7 +2449,7 @@
       "special": true
     },
     "259723": {
-      "content": "<to_sxr>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2457,7 +2457,7 @@
       "special": true
     },
     "259724": {
-      "content": "<to_szy>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2465,7 +2465,7 @@
       "special": true
     },
     "259725": {
-      "content": "<to_tao>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2473,7 +2473,7 @@
       "special": true
     },
     "259726": {
-      "content": "<to_tay>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2481,7 +2481,7 @@
       "special": true
     },
     "259727": {
-      "content": "<to_trv>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2489,7 +2489,7 @@
       "special": true
     },
     "259728": {
-      "content": "<to_tsu>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2497,7 +2497,7 @@
       "special": true
     },
     "259729": {
-      "content": "<to_xnb>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2505,7 +2505,7 @@
       "special": true
     },
     "259730": {
-      "content": "<to_xsy>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2513,6 +2513,582 @@
       "special": true
     },
     "259731": {
       "content": "<to_zh>",
       "lstrip": false,
       "normalized": false,
@@ -2753,17 +3329,24 @@
     "<dialect_hengchun>",
     "<dialect_jianhe>",
     "<dialect_junqun>",
     "<dialect_kaqun>",
     "<dialect_luanqun>",
     "<dialect_malan>",
     "<dialect_maolin>",
     "<dialect_nanwang>",
     "<dialect_northern>",
     "<dialect_sekolik>",
     "<dialect_southern>",
     "<dialect_tanqun>",
     "<dialect_tegudaya>",
     "<dialect_truku>",
     "<dialect_unknown>",
     "<dialect_wanda>",
     "<dialect_wanshan>",
@@ -2771,7 +3354,8 @@
     "<dialect_wutai>",
     "<dialect_xiqun>",
     "<dialect_xiuguluan>",
-    "<dialect_yilan_zeaol>",
     "<dialect_zeaol>",
     "<dialect_zhiben>",
     "<dialect_zhuoqun>",
@@ -2779,16 +3363,80 @@
     "<dom_culture>",
     "<dom_dictionary>",
     "<dom_essays>",
     "<dom_formosan_100_paiwan_texts>",
     "<dom_formosan_amis_myths_and_customs>",
     "<dom_formosan_epark>",
     "<dom_formosan_gitbook_translations>",
     "<dom_formosan_old_texts>",
     "<dom_formosan_paiwanstories>",
     "<dom_formosan_rik_bunun>",
     "<dom_formosan_seals>",
     "<dom_formosan_wilang_yutas_videos>",
     "<dom_formosan_yeddas_blog>",
     "<dom_formosan_zheng_data>",
     "<dom_learning_vocab>",
     "<dom_nine_level>",

       "special": true
     },
     "259652": {
+      "content": "<dialect_kanakanavu>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259653": {
+      "content": "<dialect_kaqun>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259654": {
+      "content": "<dialect_kavalan>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259655": {
+      "content": "<dialect_luanqun>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259656": {
+      "content": "<dialect_malan>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259657": {
+      "content": "<dialect_maolin>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259658": {
+      "content": "<dialect_nanwang>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259659": {
+      "content": "<dialect_northern>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259660": {
+      "content": "<dialect_saaroa>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259661": {
+      "content": "<dialect_saisiyat>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259662": {
+      "content": "<dialect_sakizaya>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259663": {
+      "content": "<dialect_sekolik>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259664": {
+      "content": "<dialect_southern>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259665": {
+      "content": "<dialect_tanqun>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259666": {
+      "content": "<dialect_tegudaya>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259667": {
+      "content": "<dialect_thao>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259668": {
+      "content": "<dialect_truku>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259669": {
+      "content": "<dialect_tsou>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259670": {
+      "content": "<dialect_unknown>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259671": {
+      "content": "<dialect_wanda>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259672": {
+      "content": "<dialect_wanshan>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259673": {
+      "content": "<dialect_wenshui>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259674": {
+      "content": "<dialect_wutai>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259675": {
+      "content": "<dialect_xiqun>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259676": {
+      "content": "<dialect_xiuguluan>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259677": {
+      "content": "<dialect_yami>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259678": {
+      "content": "<dialect_yilanzeaol>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259679": {
+      "content": "<dialect_zeaol>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259680": {
+      "content": "<dialect_zhiben>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259681": {
+      "content": "<dialect_zhuoqun>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259682": {
+      "content": "<dom_classroom_context>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259683": {
+      "content": "<dom_culture>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259684": {
+      "content": "<dom_dictionary>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259685": {
+      "content": "<dom_essays>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259686": {
+      "content": "<dom_ethnography_journal_ritual_song>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259687": {
+      "content": "<dom_formosan_100_paiwan_texts>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259688": {
+      "content": "<dom_formosan_academia_sinica_oral_legends>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259689": {
+      "content": "<dom_formosan_amis_adversative_constructions>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259690": {
+      "content": "<dom_formosan_amis_kavalan_lin_interrogative_verbs>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259691": {
+      "content": "<dom_formosan_amis_kuo_sung_comparative_constructions>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259692": {
+      "content": "<dom_formosan_amis_lifok_dongi_saopo_kimakimad>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259693": {
+      "content": "<dom_formosan_amis_myths_and_customs>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259694": {
+      "content": "<dom_formosan_amis_pa_verbs>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259695": {
+      "content": "<dom_formosan_amis_serial_verb_constructions>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259696": {
+      "content": "<dom_formosan_amis_tung_chiou>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259697": {
+      "content": "<dom_formosan_asai_sedik_language>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259698": {
+      "content": "<dom_formosan_bunun_debusser_dissertation>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259699": {
+      "content": "<dom_formosan_bunun_moriguchi_northern_texts>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259700": {
+      "content": "<dom_formosan_bunun_topic_focus>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259701": {
+      "content": "<dom_formosan_chen_fukuda_relabeling_ergative>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259702": {
+      "content": "<dom_formosan_chen_fukuda_three_ways_steal>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259703": {
+      "content": "<dom_formosan_cip_atayal_grammar_overview>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259704": {
+      "content": "<dom_formosan_dean_johnson_year_clouds_smangus>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259705": {
+      "content": "<dom_formosan_dorinda_tsai_hsiu_liu_neutral_imperfect>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259706": {
+      "content": "<dom_formosan_egerod_origin_headhunting_atayal_text_v>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259707": {
+      "content": "<dom_formosan_egerod_soren_word_order_word_classes_at>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259708": {
+      "content": "<dom_formosan_elizabeth_zeitoun_chen_huei_wu_overview>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259709": {
+      "content": "<dom_formosan_epark>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259710": {
+      "content": "<dom_formosan_ferrell_taiwan_aboriginal_groups_proble>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259711": {
+      "content": "<dom_formosan_gitbook_translations>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259712": {
+      "content": "<dom_formosan_glosbe>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259713": {
+      "content": "<dom_formosan_hala_saku_la_videos>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259714": {
+      "content": "<dom_formosan_holmer_seediq>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259715": {
+      "content": "<dom_formosan_huang_grammaticalization_squliq_atayal>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259716": {
+      "content": "<dom_formosan_huang_mei_chin_atayal_reference_grammar>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259717": {
+      "content": "<dom_formosan_huteson_rukai_survey>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259718": {
+      "content": "<dom_formosan_ilrdf_42_language_practice_word_lists>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259719": {
+      "content": "<dom_formosan_ilrdf_tousvusvutu_kita>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259720": {
+      "content": "<dom_formosan_indigenous_language_literary_awards>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259721": {
+      "content": "<dom_formosan_kanakanavu_texts>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259722": {
+      "content": "<dom_formosan_kavalan_zhang_reference_grammar>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259723": {
+      "content": "<dom_formosan_li_peirong_xu_weisheng_introduction_tru>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259724": {
+      "content": "<dom_formosan_li_peizhen_wu_xingyu_shouhu_xing>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259725": {
+      "content": "<dom_formosan_naomi_tsukida_correlative_clauses_seedi>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259726": {
+      "content": "<dom_formosan_nowbucyang_truku_thesis>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259727": {
+      "content": "<dom_formosan_old_texts>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259728": {
+      "content": "<dom_formosan_paiwan_collart_zeitoun_time_reference>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259729": {
+      "content": "<dom_formosan_paiwan_ho_five_dialects>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259730": {
+      "content": "<dom_formosan_paiwanstories>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259731": {
+      "content": "<dom_formosan_puyuma_chen_raising_to_object>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259732": {
+      "content": "<dom_formosan_puyuma_katipol_kopfjagdriten>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259733": {
+      "content": "<dom_formosan_puyuma_teng_pronominal_systems>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259734": {
+      "content": "<dom_formosan_puyuma_teng_reference_grammar>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259735": {
+      "content": "<dom_formosan_rik_bunun>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259736": {
+      "content": "<dom_formosan_robert_blust_austronesian_homeland_ling>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259737": {
+      "content": "<dom_formosan_rukai_mantauran_stories>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259738": {
+      "content": "<dom_formosan_rukai_texts>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259739": {
+      "content": "<dom_formosan_rukai_zeitoun_saying>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259740": {
+      "content": "<dom_formosan_saaroa_pan_grammar>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259741": {
+      "content": "<dom_formosan_sakizaya_affixes>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259742": {
+      "content": "<dom_formosan_seals>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259743": {
+      "content": "<dom_formosan_seediq_zhang_reference_grammar>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259744": {
+      "content": "<dom_formosan_shigeru_tsuchida_kanakanavu_texts>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259745": {
+      "content": "<dom_formosan_shigeru_tsuchida_reconstruction_proto_t>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259746": {
+      "content": "<dom_formosan_shih_rukai_adverbial>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259747": {
+      "content": "<dom_formosan_song_limei_introduction_kanakanavu_gram>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259748": {
+      "content": "<dom_formosan_song_limei_introduction_seediq_grammar>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259749": {
+      "content": "<dom_formosan_sung_kuo_descriptive_comparative_constr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259750": {
+      "content": "<dom_formosan_taoshan_elementary_school_atelier_hui_k>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259751": {
+      "content": "<dom_formosan_tsukida_naomi_verb_classification_amis>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259752": {
+      "content": "<dom_formosan_wei_matri_clan_lineage_system_ami>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259753": {
+      "content": "<dom_formosan_wilang_yutas_videos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259754": {
+      "content": "<dom_formosan_wu_jinglan_introduction_amis_grammar>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259755": {
+      "content": "<dom_formosan_yedda_palemeq_blog>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259756": {
+      "content": "<dom_formosan_yeddas_blog>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259757": {
+      "content": "<dom_formosan_yeh_meili_introduction_saisiyat_grammar>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259758": {
+      "content": "<dom_formosan_zhang_yongli_pan_jiarong_introduction_t>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259759": {
+      "content": "<dom_formosan_zheng_acl_2024>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259760": {
+      "content": "<dom_formosan_zheng_data>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259761": {
+      "content": "<dom_learning_vocab>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259762": {
+      "content": "<dom_nine_level>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259763": {
+      "content": "<dom_ntu>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259764": {
+      "content": "<dom_picture_book>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259765": {
+      "content": "<dom_picture_story>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259766": {
+      "content": "<dom_presidential_apology>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259767": {
+      "content": "<dom_reading_writing>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259768": {
+      "content": "<dom_unknown>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259769": {
+      "content": "<dom_youtube>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259770": {
+      "content": "<src_ami>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259771": {
+      "content": "<src_bnn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259772": {
+      "content": "<src_ckv>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259773": {
+      "content": "<src_dru>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259774": {
+      "content": "<src_eng>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259775": {
+      "content": "<src_pwn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259776": {
+      "content": "<src_pyu>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259777": {
+      "content": "<src_ssf>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259778": {
+      "content": "<src_sxr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259779": {
+      "content": "<src_szy>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259780": {
+      "content": "<src_tao>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259781": {
+      "content": "<src_tay>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259782": {
+      "content": "<src_trv>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259783": {
+      "content": "<src_tsu>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259784": {
+      "content": "<src_xnb>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259785": {
+      "content": "<src_xsy>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259786": {
+      "content": "<src_zh>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259787": {
+      "content": "<to_ami>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259788": {
+      "content": "<to_bnn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259789": {
+      "content": "<to_ckv>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259790": {
+      "content": "<to_dru>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259791": {
+      "content": "<to_eng>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259792": {
+      "content": "<to_pwn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259793": {
+      "content": "<to_pyu>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259794": {
+      "content": "<to_ssf>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259795": {
+      "content": "<to_sxr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259796": {
+      "content": "<to_szy>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259797": {
+      "content": "<to_tao>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259798": {
+      "content": "<to_tay>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259799": {
+      "content": "<to_trv>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259800": {
+      "content": "<to_tsu>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259801": {
+      "content": "<to_xnb>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259802": {
+      "content": "<to_xsy>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259803": {
       "content": "<to_zh>",
       "lstrip": false,
       "normalized": false,
     "<dialect_hengchun>",
     "<dialect_jianhe>",
     "<dialect_junqun>",
+    "<dialect_kanakanavu>",
     "<dialect_kaqun>",
+    "<dialect_kavalan>",
     "<dialect_luanqun>",
     "<dialect_malan>",
     "<dialect_maolin>",
     "<dialect_nanwang>",
     "<dialect_northern>",
+    "<dialect_saaroa>",
+    "<dialect_saisiyat>",
+    "<dialect_sakizaya>",
     "<dialect_sekolik>",
     "<dialect_southern>",
     "<dialect_tanqun>",
     "<dialect_tegudaya>",
+    "<dialect_thao>",
     "<dialect_truku>",
+    "<dialect_tsou>",
     "<dialect_unknown>",
     "<dialect_wanda>",
     "<dialect_wanshan>",
     "<dialect_wutai>",
     "<dialect_xiqun>",
     "<dialect_xiuguluan>",
+    "<dialect_yami>",
+    "<dialect_yilanzeaol>",
     "<dialect_zeaol>",
     "<dialect_zhiben>",
     "<dialect_zhuoqun>",
     "<dom_culture>",
     "<dom_dictionary>",
     "<dom_essays>",
+    "<dom_ethnography_journal_ritual_song>",
     "<dom_formosan_100_paiwan_texts>",
+    "<dom_formosan_academia_sinica_oral_legends>",
+    "<dom_formosan_amis_adversative_constructions>",
+    "<dom_formosan_amis_kavalan_lin_interrogative_verbs>",
+    "<dom_formosan_amis_kuo_sung_comparative_constructions>",
+    "<dom_formosan_amis_lifok_dongi_saopo_kimakimad>",
     "<dom_formosan_amis_myths_and_customs>",
+    "<dom_formosan_amis_pa_verbs>",
+    "<dom_formosan_amis_serial_verb_constructions>",
+    "<dom_formosan_amis_tung_chiou>",
+    "<dom_formosan_asai_sedik_language>",
+    "<dom_formosan_bunun_debusser_dissertation>",
+    "<dom_formosan_bunun_moriguchi_northern_texts>",
+    "<dom_formosan_bunun_topic_focus>",
+    "<dom_formosan_chen_fukuda_relabeling_ergative>",
+    "<dom_formosan_chen_fukuda_three_ways_steal>",
+    "<dom_formosan_cip_atayal_grammar_overview>",
+    "<dom_formosan_dean_johnson_year_clouds_smangus>",
+    "<dom_formosan_dorinda_tsai_hsiu_liu_neutral_imperfect>",
+    "<dom_formosan_egerod_origin_headhunting_atayal_text_v>",
+    "<dom_formosan_egerod_soren_word_order_word_classes_at>",
+    "<dom_formosan_elizabeth_zeitoun_chen_huei_wu_overview>",
     "<dom_formosan_epark>",
+    "<dom_formosan_ferrell_taiwan_aboriginal_groups_proble>",
     "<dom_formosan_gitbook_translations>",
+    "<dom_formosan_glosbe>",
+    "<dom_formosan_hala_saku_la_videos>",
+    "<dom_formosan_holmer_seediq>",
+    "<dom_formosan_huang_grammaticalization_squliq_atayal>",
+    "<dom_formosan_huang_mei_chin_atayal_reference_grammar>",
+    "<dom_formosan_huteson_rukai_survey>",
+    "<dom_formosan_ilrdf_42_language_practice_word_lists>",
+    "<dom_formosan_ilrdf_tousvusvutu_kita>",
+    "<dom_formosan_indigenous_language_literary_awards>",
+    "<dom_formosan_kanakanavu_texts>",
+    "<dom_formosan_kavalan_zhang_reference_grammar>",
+    "<dom_formosan_li_peirong_xu_weisheng_introduction_tru>",
+    "<dom_formosan_li_peizhen_wu_xingyu_shouhu_xing>",
+    "<dom_formosan_naomi_tsukida_correlative_clauses_seedi>",
+    "<dom_formosan_nowbucyang_truku_thesis>",
     "<dom_formosan_old_texts>",
+    "<dom_formosan_paiwan_collart_zeitoun_time_reference>",
+    "<dom_formosan_paiwan_ho_five_dialects>",
     "<dom_formosan_paiwanstories>",
+    "<dom_formosan_puyuma_chen_raising_to_object>",
+    "<dom_formosan_puyuma_katipol_kopfjagdriten>",
+    "<dom_formosan_puyuma_teng_pronominal_systems>",
+    "<dom_formosan_puyuma_teng_reference_grammar>",
     "<dom_formosan_rik_bunun>",
+    "<dom_formosan_robert_blust_austronesian_homeland_ling>",
+    "<dom_formosan_rukai_mantauran_stories>",
+    "<dom_formosan_rukai_texts>",
+    "<dom_formosan_rukai_zeitoun_saying>",
+    "<dom_formosan_saaroa_pan_grammar>",
+    "<dom_formosan_sakizaya_affixes>",
     "<dom_formosan_seals>",
+    "<dom_formosan_seediq_zhang_reference_grammar>",
+    "<dom_formosan_shigeru_tsuchida_kanakanavu_texts>",
+    "<dom_formosan_shigeru_tsuchida_reconstruction_proto_t>",
+    "<dom_formosan_shih_rukai_adverbial>",
+    "<dom_formosan_song_limei_introduction_kanakanavu_gram>",
+    "<dom_formosan_song_limei_introduction_seediq_grammar>",
+    "<dom_formosan_sung_kuo_descriptive_comparative_constr>",
+    "<dom_formosan_taoshan_elementary_school_atelier_hui_k>",
+    "<dom_formosan_tsukida_naomi_verb_classification_amis>",
+    "<dom_formosan_wei_matri_clan_lineage_system_ami>",
     "<dom_formosan_wilang_yutas_videos>",
+    "<dom_formosan_wu_jinglan_introduction_amis_grammar>",
+    "<dom_formosan_yedda_palemeq_blog>",
     "<dom_formosan_yeddas_blog>",
+    "<dom_formosan_yeh_meili_introduction_saisiyat_grammar>",
+    "<dom_formosan_zhang_yongli_pan_jiarong_introduction_t>",
+    "<dom_formosan_zheng_acl_2024>",
     "<dom_formosan_zheng_data>",
     "<dom_learning_vocab>",
     "<dom_nine_level>",