---
license: apache-2.0
language:
  - zh
  - en
tags:
  - text-to-speech
  - tts
  - onnx
  - on-device
  - jetson
  - telephony
  - mandarin
  - taiwanese-mandarin
base_model: owensong/Inflect-Nano-v1
library_name: onnxruntime
pipeline_tag: text-to-speech
---

# PrimeTTS — tiny bilingual zh-TW + English TTS (8 kHz, CPU)

**PrimeTTS** is an ultra-small **4.63M-parameter** Mandarin (Taiwan) + English text-to-speech model
that runs **entirely on CPU** via `onnxruntime` and emits **8 kHz** audio — sized for **G.711
telephony** and **on-device (Jetson-class)** deployment.

It is a **fine-tune of [`owensong/Inflect-Nano-v1`](https://huggingface.co/owensong/Inflect-Nano-v1)**
(Apache-2.0) — the **same, frozen architecture**, retrained for zh-TW + English. No architecture
changes, no neural-architecture-search.

- **Acoustic** (`MicroFastSpeech`, ~3.47M): FastSpeech-style depthwise Conv-FFN (**no attention**),
  external durations + length regulator, frame-pitch, BiGRU, postnet.
- **Vocoder** (~1.17M): Snake-HiFiGAN. We added an **8 kHz variant** (`snake_8k`: sr 8000, n_fft 512,
  hop 128, 80 mels) and a **zh-TW + English frontend** (bopomofo + arpabet, one unified phone sequence).

**Live demo:** https://huggingface.co/spaces/Luigi/inflect-nano-zhtw-en-8k-demo

---

## How these weights were obtained — the two things that matter

Inflect-Nano-v1's 4.63M architecture is **not capacity-limited** for this task. We confirmed the
original English checkpoint scores ~0.05 CER on our eval, yet our first retrains were unintelligible
(~0.88 CER). The gap was **not** the model size. Two fixes — **architecture frozen** — took held-out
Mandarin CER from **~0.88 → ~0.06**:

1. **Phone-level forced alignment.** FastSpeech needs a per-phone duration for every training clip.
   Crude char/letter-CTC alignment (then splitting a char's span across its phones by heuristic) gives
   *wrong relative phone durations* → the acoustic learns a mis-timed phone→mel map → over-smoothed,
   garbled output in **every** language. Replace it with **true phone-level forced alignment**
   (`align_durations_v4.py`: the espeak phoneme-CTC model `facebook/wav2vec2-lv-60-espeak-cv-ft` +
   `torchaudio.forced_align`, aligning *your frontend's own phone sequence* mapped to IPA). This alone:
   Mandarin 0.88 → 0.40.

2. **Diverse, well-covered training text.** The model can only pronounce characters/words it has seen.
   A narrow corpus (≈234 Han chars) left ~39% of held-out characters unseen. Expanding character/word
   coverage (`select_diverse_text.py`: Tatoeba → OpenCC `s2twp` → greedy coverage) took Mandarin
   0.40 → **0.06**. Applied symmetrically to English, the same recipe yields a genuinely **bilingual**
   model (zh-only ≈ 0.13, English ≈ 0.16) in one 4.63M net — no language routing.

Everything else (loss weights, steps, etc.) is essentially Inflect-Nano-v1's defaults.

---

## Reproduce / fine-tune (your own voice or language)

```
scripts/        full pipeline (frontend, aligner, corpus-gen, train, export, eval)
inflect_nano/   the trainer (acoustic.py + vocoder.py), forked from Inflect-Nano-v1 (LICENSE included)
weights/        deployable ONNX (encoder/decoder/vocoder) + meta.json + symbol_table.json
                + acoustic_zh_v2_60k.pt (PyTorch checkpoint, to resume / fine-tune)
```

**Step 1 — Teacher corpus.** Synthesize (text → audio) clips in ONE target voice, ASR-gated for
fidelity. We used **BreezyVoice** (MediaTek, CosyVoice-based; single "mark" voice, does zh + en) with a
**Breeze-ASR-25** gate (`gen_breezy_corpus.py` — keeps a clip only if its ASR transcript matches the
intended text, t2s-normalized). Any clean single-speaker TTS/recordings work.

**Step 2 — Diverse text.** `python scripts/select_diverse_text.py --lang zh --n 6000 --out zh.tsv`
(and `--lang en`). Feed the `.tsv` to step 1. Coverage is the single biggest driver of held-out quality.

**Step 3 — Phone-level alignment.** `python scripts/align_durations_v4.py --manifest corpus/manifest.jsonl
--out align.jsonl` → per-phone durations from real audio. **This is the key fix — do not skip it.**

**Step 4 — Train acoustic.** `inflect_nano.acoustic --durations-jsonl align.jsonl --vocoder-variant
snake_8k --sample-rate 8000 ...` (see `scripts/run_bilingual.sh`). Mix languages in one corpus, single
speaker, for native code-mix. ~60k steps, batch 16.

**Step 5 — Train vocoder.** `inflect_nano.vocoder --train-jsonl voc_rows.jsonl --variant snake_8k ...`
(see `scripts/run_voc_retrain.sh`). Train on the same diverse audio; higher `--stft-weight` = crisper.

**Step 6 — Export ONNX.** `python scripts/export_8k.py --acoustic-ckpt <pt> --vocoder-ckpt <pt>
--out-dir onnx/` → `acoustic_encoder.onnx` → numpy length-regulator (`host_regulate`) →
`acoustic_decoder.onnx` → `vocoder.onnx`. Fully torch-free at inference.

**Step 7 — Evaluate.** `synth_from_text.py` + `assess_big.py` (offline X-ASR CER/WER). Use ≥30 held-out
sentences — small eval sets are too noisy.

### Inference (torch-free, CPU)
```python
import frontend_bopomofo as F, numpy as np, onnxruntime as ort
o = F.text_to_ids("您好,歡迎使用 PrimeTTS。")     # bopomofo + arpabet -> ids
# encoder.onnx -> host_regulate (numpy) -> decoder.onnx -> vocoder.onnx -> 8 kHz wav
```
See `scripts/synth_from_text.py` for the full ~40-line runtime (also runs as-is on a Jetson Nano CPU).

---

## Credits & licenses

- **Base model / trainer:** [`owensong/Inflect-Nano-v1`](https://huggingface.co/owensong/Inflect-Nano-v1)
  (Apache-2.0; `inflect_nano/LICENSE.inflect-nano`).
- **Teacher:** BreezyVoice (MediaTek Research). **Gate ASR:** `Breeze-ASR-25` (MediaTek Research).
- **Aligner:** `facebook/wav2vec2-lv-60-espeak-cv-ft` + `torchaudio.forced_align`.
- **Frontend:** `g2pw` (Taiwan bopomofo, polyphone disambiguation) + `g2p_en` (arpabet).
- **Eval ASR:** sherpa-onnx X-ASR (zh-en Zipformer). **Text:** Tatoeba (CC-BY 2.0 FR).

This repository: **Apache-2.0**.