Release 1.0.0: Chinese TTS (Inflect-Micro-v2 arch + BigVGAN), ONNX + AX650 axmodel + checkpoints

.gitattributes

@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+models/acoustic_female.onnx.data filter=lfs diff=lfs merge=lfs -text
+models/acoustic_male.onnx.data filter=lfs diff=lfs merge=lfs -text
+models/bigvgan_base.axmodel filter=lfs diff=lfs merge=lfs -text
+models/bigvgan_base.onnx.data filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,101 @@
+---
+license: apache-2.0
+language:
+- zh
+tags:
+- text-to-speech
+- tts
+- chinese
+- axera
+- axmodel
+- onnx
+pipeline_tag: text-to-speech
+---
+
+# Inflect-Micro-v2-zh
+
+从零训练的中文 TTS：基于 [Inflect-Micro-v2](https://huggingface.co/owensong/Inflect-Micro-v2)
+架构（Apache-2.0）+ [NVIDIA BigVGAN](https://github.com/NVIDIA/BigVGAN) 声码器（MIT）。
+声学模型 6.7M 参数（text→100-band mel），可在 **AX650 系列 NPU** 上实时部署
+（CPU + NPU 混合架构，RTF 0.15-0.19）。
+
+## 文件说明
+
+```
+models/
+├── acoustic_female.onnx (+ .data)   # 女声 文本→mel（CPU fp32，ls=0.7）
+├── acoustic_male.onnx   (+ .data)   # 男声 文本→mel（CPU fp32，ls=0.85）
+├── bigvgan_base.onnx    (+ .data)   # 声码器 mel→wav（CPU fp32，动态帧）
+├── bigvgan_base.axmodel              # 声码器 mel→wav（AX650 NPU int8）
+└── model_meta.json                   # 模型规格元数据
+checkpoints/                          # 训练权重（PyTorch）
+├── base300.pt                        # 基础模型（AiShell-3，20 说话人）
+├── female_gan.pt                     # 女声（CosyVoice 音色 + GAN）
+└── male_gan.pt                       # 男声（SSB0609 音色 + GAN）
+python/                               # Python 推理 SDK
+reports/                              # 验证报告（可懂度/性能/量化）
+```
+
+## 快速开始（Python，CPU）
+
+```bash
+pip install onnxruntime numpy soundfile pypinyin
+
+python python/infer_onnx.py \
+  --text "你好世界，今天天气真不错，适合出去散步。" \
+  --acoustic models/acoustic_female.onnx \
+  --vocoder models/bigvgan_base.onnx \
+  --output out.wav
+```
+
+## 快速开始（AX650 板，CPU+NPU 混合）
+
+板上安装 `onnxruntime`、`pypinyin` 和 `axengine`（AXERA 提供）：
+
+```bash
+python python/infer_board.py \
+  --text "人工智能技术正在改变我们的生活方式。" \
+  --acoustic models/acoustic_female.onnx \
+  --vocoder models/bigvgan_base.axmodel \
+  --output out.wav
+```
+
+`infer_board.py` 自动完成：文本分句 → 声学模型（CPU）→ mel 后处理
+（静音门限 + 句尾拉伸）→ 声码器（NPU）分块拼接。
+
+## 性能（AX650N 实测）
+
+| 组件 | 耗时 | 说明 |
+|---|---|---|
+| acoustic（CPU fp32） | 0.30s/句 | 静态 max_mel=1024（单句 ≤10.9s） |
+| vocoder（NPU int8） | 0.30s/块 | 512 mel 帧 = 5.46s 音频 |
+| 整链路 RTF | 0.15-0.19 | 约 6x 实时 |
+
+## 可懂度（SenseVoice，8 句测试）
+
+| 音色 | CER |
+|---|---|
+| 女声（CPU 全 fp32） | 0.075 |
+| 男声（CPU 全 fp32） | 0.108 |
+
+## 说明
+
+- 声学模型必须 CPU fp32：VITS 的 flow/duration 对 NPU 8/16-bit 量化系统性敏感
+  （int8/U16/BRecQ 全部验证失败）；声码器 NPU int8 可用
+- 长文本自动分句（≤250 tokens/句）+ 声码器分块拼接（重叠交叉淡化）
+- 训练流程与复现脚本见 [GitHub 仓库](https://github.com/ml-inory/Inflect-Micro-v2-zh)
+
+## 致谢
+
+- [owensong/Inflect-Micro-v2](https://huggingface.co/owensong/Inflect-Micro-v2)（架构，Apache-2.0）
+- [NVIDIA BigVGAN](https://github.com/NVIDIA/BigVGAN)（声码器，MIT）
+- [FunAudioLLM/CosyVoice](https://github.com/FunAudioLLM/CosyVoice)（数据生成，Apache-2.0）
+- [OpenSLR AISHELL-3](https://www.openslr.org/resources/93/)（中文语料）
+- AXERA Pulsar2（AX 编译工具链）
+
+完整第三方声明见 GitHub 仓库 `THIRD_PARTY_NOTICES.md`。
+
+## License
+
+[Apache-2.0](https://github.com/ml-inory/Inflect-Micro-v2-zh/blob/main/LICENSE)
+（与原版 Inflect-Micro-v2 一致）

checkpoints/base300.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ddd749d15dda2d378a362314bd6bc61284edc1dec8ff99c4bec58b84ae813b3b
+size 313040243

checkpoints/female_gan.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc72f8e9ebf1f5728c259cc05f98d52b2d2096ec031fb4c7274d7fa420551bfd
+size 26926933

checkpoints/male_gan.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e5873d35dfc6d690d5ce44079d5c4c61d6bbc12e320afc2eacd25232fbc5daff
+size 26926933

models/acoustic_female.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8264431335447b241d0e6f30a46e9f47e94b93285fc5ee269f8676a8a3157c55
+size 990534

models/acoustic_female.onnx.data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:074985877003699d58897b03fd84d3c95c19d7fc59855465f9269510145cfeec
+size 35846400

models/acoustic_male.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5a250387759f7417c15d236d9e75201c012d3c9bacef1a4ecf44ef33ca9dc230
+size 989998

models/acoustic_male.onnx.data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:033a94d8717b874fdb437c3569cc5583fb1d7bd123956d0d0dac978cb2b04017
+size 35846400

models/bigvgan_base.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:44d12f6a8ceca75462873603405cd789f4f71261157ef28c31a718718238cb79
+size 27678831

models/bigvgan_base.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:77d1fe4b14242776253713694f8d8abd2189551eb7d8d639f5893fc1553f61ae
+size 1589797

models/bigvgan_base.onnx.data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a5fc77bcc63b7b1ff8a62f2d410408067c013df14e7269c88e4a60c09b8c67fb
+size 56754176

models/model_meta.json

@@ -0,0 +1,75 @@
+{
+  "name": "Inflect-Micro-v2-zh",
+  "version": "1.0.0",
+  "description": "Chinese TTS: text-to-mel acoustic model (6.7M) + NVIDIA BigVGAN vocoder",
+  "pipeline": ["text", "mel", "wav"],
+  "sampling_rate": 24000,
+  "mel": {
+    "n_mels": 100,
+    "n_fft": 1024,
+    "hop_length": 256,
+    "win_length": 1024,
+    "fmin": 0,
+    "fmax": 12000,
+    "style": "bigvgan"
+  },
+  "models": {
+    "acoustic_female.onnx": {
+      "role": "text->mel (female voice)",
+      "backend": "CPU (onnxruntime, fp32)",
+      "inputs": {
+        "x": "int64[1,256] token ids (with blanks, max 256)",
+        "x_lengths": "int64[1]",
+        "noise_z": "float32[1,192,1024]"
+      },
+      "outputs": {
+        "mel": "float32[1,100,1024] log-mel",
+        "y_lengths": "int32[1] valid mel frames"
+      },
+      "length_scale": 0.7,
+      "max_text": 256,
+      "max_mel": 1024
+    },
+    "acoustic_male.onnx": {
+      "role": "text->mel (male voice)",
+      "backend": "CPU (onnxruntime, fp32)",
+      "inputs": {
+        "x": "int64[1,256]",
+        "x_lengths": "int64[1]",
+        "noise_z": "float32[1,192,1024]"
+      },
+      "outputs": {
+        "mel": "float32[1,100,1024]",
+        "y_lengths": "int32[1]"
+      },
+      "length_scale": 0.85,
+      "max_text": 256,
+      "max_mel": 1024
+    },
+    "bigvgan_base.onnx": {
+      "role": "mel->wav (vocoder, dynamic frames)",
+      "backend": "CPU (onnxruntime, fp32)",
+      "inputs": {"mel": "float32[1,100,T]"},
+      "outputs": {"wav": "float32[1,1,T*256]"}
+    },
+    "bigvgan_base.axmodel": {
+      "role": "mel->wav (vocoder, AX650 NPU)",
+      "backend": "NPU (axengine, int8, AX650/NPU3)",
+      "inputs": {"mel": "float32[1,100,512]"},
+      "outputs": {"wav": "float32[1,1,131072]"},
+      "compiled_by": "Pulsar2 7.0"
+    }
+  },
+  "performance": {
+    "ax650_cpu_acoustic_seconds": 0.30,
+    "ax650_npu_vocoder_seconds_per_block": 0.30,
+    "rtf_mixed_cpu_npu": "0.15-0.19"
+  },
+  "license": "apache-2.0",
+  "acknowledgements": [
+    "Inflect-Micro-v2 (owensong, Apache-2.0)",
+    "NVIDIA BigVGAN (MIT)",
+    "FunAudioLLM CosyVoice (Apache-2.0)",
+    "OpenSLR AISHELL-3"
+  ]
+}

python/cn_frontend/__init__.py

@@ -0,0 +1,2 @@
+from .symbols import symbols, _symbol_to_id, get_phoneme_ids, PAD_ID, SPACE_ID
+from .pinyin_to_sequence import text_to_sequence, pinyin_to_ids

python/cn_frontend/pinyin_to_sequence.py

@@ -0,0 +1,27 @@
+"""Convert Chinese text to pinyin character sequence."""
+
+import re
+from pypinyin import lazy_pinyin, Style
+from .symbols import get_phoneme_ids
+
+_whitespace_re = re.compile(r"\s+")
+
+
+def text_to_sequence(text: str) -> list[int]:
+    """Chinese text -> pinyin character IDs.
+
+    Example: "你好" -> "ni3 hao3" -> [n, i, 3, <space>, h, a, o, 3]
+    """
+    text = re.sub(_whitespace_re, "", text)
+
+    # Convert to pinyin with tone digits
+    py_list = lazy_pinyin(text, style=Style.TONE3, neutral_tone_with_five=True)
+
+    # Join with spaces between words
+    pinyin_str = " ".join(py_list)
+
+    return get_phoneme_ids(pinyin_str)
+
+
+def pinyin_to_ids(pinyin_str: str) -> list[int]:
+    return get_phoneme_ids(pinyin_str)

python/cn_frontend/symbols.py

@@ -0,0 +1,20 @@
+"""Chinese pinyin character-level symbol set."""
+
+_pad = "_"
+
+# All possible pinyin characters (a-z) plus tone digits (1-5)
+_pinyin_chars = "abcdefghijklmnopqrstuvwxyz12345 "
+
+# Chinese punctuation
+_punct = "，。！？；：、…—""''（）《》·"
+
+symbols = [_pad] + list(_pinyin_chars) + list(_punct)
+
+_symbol_to_id = {s: i for i, s in enumerate(symbols)}
+
+PAD_ID = 0
+SPACE_ID = _symbol_to_id.get(" ", len(symbols))
+
+
+def get_phoneme_ids(phonemes: str) -> list[int]:
+    return [_symbol_to_id.get(p, PAD_ID) for p in phonemes]

python/infer_board.py

@@ -0,0 +1,247 @@
+"""AX board inference: acoustic (CPU/ONNX) + BigVGAN (NPU/axmodel).
+
+Run on the AX board with:
+  python3 infer_board.py --text "你好" --acoustic acoustic_female.onnx \
+      --vocoder bigvgan_base.axmodel --output out.wav
+"""
+
+import argparse
+import os
+import sys
+
+import numpy as np
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from cn_frontend import text_to_sequence
+
+MAX_TEXT = 256
+MAX_MEL = 1024
+VOC_CHUNK = 512        # BigVGAN 静态输入帧数（5.46s@24kHz）
+VOC_OVERLAP = 48       # 分块重叠帧（0.5s），交叉淡化消除边界爆音
+MAX_SENT_TOKENS = 250  # 单句 token 上限（acoustic 上限 256，留 6 余量）
+MEL_GATE_LO = -3.5     # mel 弱帧门限：低于此压到 floor（消除静音段底噪）
+MEL_GATE_HI = -2.2     # 过渡区上界，高于此帧保持不动
+LENGTH_SCALE = 0.7     # 语速（音节时长校正，接近自然节奏）
+TAIL_STRETCH_OLD = 20  # 句尾拉伸：最后 N 帧
+TAIL_STRETCH_NEW = 45  # 拉伸到 N 帧（解决句尾音节过短/尾字被吞）
+
+
+def load_acoustic(path):
+    """Load acoustic ONNX (CPU, fp32)."""
+    import onnxruntime as ort
+    so = ort.SessionOptions()
+    so.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_ALL
+    return ort.InferenceSession(path, so, providers=["CPUExecutionProvider"])
+
+
+def load_vocoder(path):
+    """Load BigVGAN axmodel (NPU via axengine)."""
+    if path.endswith(".onnx"):
+        import onnxruntime as ort
+        so = ort.SessionOptions()
+        so.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_ALL
+        return ort.InferenceSession(path, so, providers=["CPUExecutionProvider"])
+    import axengine as axe
+    try:
+        return axe.InferenceSession(path, providers=["AxEngineExecutionProvider"])
+    except Exception:
+        return axe.InferenceSession(
+            path, providers=["RemoteAXExecutionProvider"],
+            provider_options={"host": os.environ.get("AX_BOARD", "127.0.0.1"),
+                              "port": "18500"})
+
+
+def split_sentences(text, max_tokens=MAX_SENT_TOKENS):
+    """Split long text into sentences that fit the acoustic model."""
+    ids = text_to_sequence(text)
+    tokens = [0]
+    for pid in ids:
+        tokens.append(pid)
+        tokens.append(0)
+    if len(tokens) <= max_tokens:
+        return [text]
+    # split on punctuation, then on hard limit
+    import re
+    parts = re.split(r"([，。！？；：、,.!?;:])", text)
+    sents, cur = [], ""
+    for p in parts:
+        cand = cur + p
+        cand_tokens = len([0]) + 2 * len(text_to_sequence(cand))
+        if len(text_to_sequence(cand)) * 2 + 1 > max_tokens and cur:
+            sents.append(cur)
+            cur = p
+        else:
+            cur = cand
+    if cur:
+        sents.append(cur)
+    # hard fallback: cut by characters
+    out = []
+    for s in sents:
+        while len(text_to_sequence(s)) * 2 + 1 > max_tokens:
+            cut = max(1, (max_tokens - 1) // 2)
+            out.append(s[:cut])
+            s = s[cut:]
+        if s:
+            out.append(s)
+    return [s for s in out if s.strip()]
+
+
+def text_to_inputs(text, noise_scale, seed):
+    np.random.seed(seed)
+    ids = text_to_sequence(text)
+    tokens = [0]
+    for pid in ids:
+        tokens.append(pid)
+        tokens.append(0)
+    n = min(len(tokens), MAX_TEXT)
+    x = np.zeros((1, MAX_TEXT), dtype=np.int64)
+    x[0, :n] = tokens[:n]
+    x_lengths = np.array([n], dtype=np.int64)
+    noise_z = np.random.randn(1, 192, MAX_MEL).astype(np.float32)
+    if noise_scale != 0.3:
+        noise_z = noise_z * (noise_scale / 0.3)
+    return x, x_lengths, noise_z
+
+
+def acoustic_to_mel(acoustic, x, x_lengths, noise_z):
+    mel, y_lengths = acoustic.run(None, {
+        "x": x, "x_lengths": x_lengths, "noise_z": noise_z,
+    })
+    return mel[:, :, :int(y_lengths[0])]
+
+
+def mel_soft_gate(mel, thr_lo=MEL_GATE_LO, thr_hi=MEL_GATE_HI, floor=-11.5):
+    """压平预测 mel 的弱帧（静音/停顿）以消除渲染底噪。
+    只调整帧能量 < thr_lo 的帧（向 floor 收敛）和过渡区，
+    强语音帧（>= thr_hi）完全保持。"""
+    out = mel.copy()
+    fe = out.max(axis=1)
+    for i in range(mel.shape[2]):
+        f = fe[0, i]
+        if f < thr_lo:
+            target = floor
+        elif f < thr_hi:
+            t = (f - thr_lo) / (thr_hi - thr_lo)
+            target = floor * (1 - t) + f * t
+        else:
+            continue
+        out[:, :, i] = out[:, :, i] - f + target
+    return out
+
+
+def tail_stretch(mel, n_old=TAIL_STRETCH_OLD, n_new=TAIL_STRETCH_NEW):
+    """句尾 mel 拉伸：最后 n_old 帧线性插值到 n_new 帧。
+    模型对句尾音节 duration 预测偏短（如 11 帧 vs 参考 43 帧），
+    拉伸后尾字清晰完整。"""
+    T = mel.shape[2]
+    if T <= n_old:
+        return mel
+    tail = mel[:, :, -n_old:]
+    xo = np.linspace(0, 1, n_old)
+    xn = np.linspace(0, 1, n_new)
+    nt = np.stack([np.interp(xn, xo, tail[0, c]) for c in range(mel.shape[1])])
+    return np.concatenate([mel[:, :, :-n_old], nt[None]], axis=2)
+
+
+def vocoder_chunked(vocoder, mel):
+    """Run BigVGAN on arbitrary-length mel via overlapped chunks + crossfade."""
+    mel = mel.astype(np.float32)
+    T = mel.shape[2]
+    if T <= VOC_CHUNK:
+        m = np.pad(mel, ((0, 0), (0, 0), (0, VOC_CHUNK - T)),
+                   constant_values=-11.5)
+        return vocoder.run(None, {"mel": m})[0][0, 0]
+    hop = VOC_CHUNK - VOC_OVERLAP
+    starts = list(range(0, T, hop))
+    frame = 256
+    wav = np.zeros(T * frame)
+    for i, s in enumerate(starts):
+        v = min(VOC_CHUNK, T - s)
+        block = mel[:, :, s:s + VOC_CHUNK]
+        if block.shape[2] < VOC_CHUNK:
+            block = np.pad(block, ((0, 0), (0, 0), (0, VOC_CHUNK - block.shape[2])),
+                           constant_values=-11.5)
+        c = vocoder.run(None, {"mel": block})[0][0, 0]
+        c = c[:v * frame]
+        base = s * frame
+        ov = 0
+        if i > 0:
+            ov = min(VOC_OVERLAP * frame, base, len(c))
+            fade = np.linspace(0, 1, ov)
+            wav[base - ov:base] = wav[base - ov:base] * (1 - fade) + c[:ov] * fade
+        wav[base + ov:base + len(c)] = c[ov:]
+    return wav
+
+
+def compress_pauses(wav, sr=24000, min_pause_ms=90, target_ms=60):
+    """Compress long silences to reduce choppy rhythm."""
+    win, hop = int(sr * 0.01), int(sr * 0.005)
+    n = (len(wav) - win) // hop
+    e = np.array([np.sqrt(np.mean(wav[i*hop:i*hop+win]**2)) for i in range(n)])
+    thr = max(e.max() * 0.12, 0.008)
+    sil = e < thr
+    # find silence runs
+    runs = []
+    i = 0
+    while i < len(sil):
+        if sil[i]:
+            j = i
+            while j < len(sil) and sil[j]:
+                j += 1
+            runs.append((i, j))
+            i = j
+        else:
+            i += 1
+    keep = np.ones(len(wav), dtype=bool)
+    for s, e_ in runs:
+        dur_ms = (e_ - s) * hop / sr * 1000
+        if dur_ms > min_pause_ms:
+            s0, s1 = s * hop, min(e_ * hop + win, len(wav))
+            target = int(target_ms / 1000 * sr)
+            # keep first `target` samples of the pause
+            if s1 - s0 > target:
+                keep[s0 + target:s1] = False
+    out = wav[keep]
+    return out
+
+
+def synthesize(acoustic, vocoder, text, noise_scale=0.3, seed=0):
+    sents = split_sentences(text)
+    pieces = []
+    for s in sents:
+        x, x_lengths, noise_z = text_to_inputs(s, noise_scale, seed)
+        mel = acoustic_to_mel(acoustic, x, x_lengths, noise_z)
+        mel = mel_soft_gate(mel)
+        mel = tail_stretch(mel)
+        pieces.append(vocoder_chunked(vocoder, mel))
+    if len(pieces) == 1:
+        return pieces[0], pieces[0].shape[0] // 256
+    # concat sentences with a short pause
+    gap = np.zeros(int(24000 * 0.2))
+    wav = pieces[0]
+    for p in pieces[1:]:
+        wav = np.concatenate([wav, gap, p])
+    return wav, wav.shape[0] // 256
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--text", required=True)
+    parser.add_argument("--acoustic", default="export/acoustic_female.onnx")
+    parser.add_argument("--vocoder", default="export/axmodel/bigvgan_base.axmodel")
+    parser.add_argument("--output", default="board_out.wav")
+    parser.add_argument("--noise_scale", type=float, default=0.3)
+    args = parser.parse_args()
+
+    import soundfile as sf
+    acoustic = load_acoustic(args.acoustic)
+    vocoder = load_vocoder(args.vocoder)
+    wav, T = synthesize(acoustic, vocoder, args.text,
+                        noise_scale=args.noise_scale)
+    wav = wav / (np.abs(wav).max() + 1e-8) * 0.95
+    sf.write(args.output, wav, 24000)
+    print(f"saved: {args.output} ({len(wav)/24000:.2f}s, mel_frames={T})")
+
+
+if __name__ == "__main__":
+    main()

python/infer_onnx.py

@@ -0,0 +1,50 @@
+"""End-to-end ONNX inference: text -> acoustic.onnx -> mel -> bigvgan.onnx -> wav."""
+
+import argparse
+import os
+import sys
+
+import numpy as np
+import onnxruntime as ort
+import soundfile as sf
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from cn_frontend import text_to_sequence
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--text", required=True)
+    parser.add_argument("--acoustic", default="export/acoustic_female.onnx")
+    parser.add_argument("--vocoder", default="export/bigvgan_base.onnx")
+    parser.add_argument("--output", default="export/onnx_demo.wav")
+    parser.add_argument("--max_text", type=int, default=256)
+    parser.add_argument("--max_mel", type=int, default=2048)
+    parser.add_argument("--noise_scale", type=float, default=0.3)
+    parser.add_argument("--seed", type=int, default=0)
+    args = parser.parse_args()
+
+    from infer_board import split_sentences, text_to_inputs, tail_stretch
+    ac = ort.InferenceSession(args.acoustic, providers=["CPUExecutionProvider"])
+    vc = ort.InferenceSession(args.vocoder, providers=["CPUExecutionProvider"])
+
+    pieces = []
+    for s in split_sentences(args.text):
+        x, x_lengths, noise_z = text_to_inputs(s, args.noise_scale, args.seed)
+        mel, y_lengths = ac.run(None, {
+            "x": x, "x_lengths": x_lengths, "noise_z": noise_z,
+        })
+        T = int(y_lengths[0])
+        mel = mel[:, :, :T]
+        mel = tail_stretch(mel)
+        print(f"  sentence mel: {mel.shape} (y_lengths={T})")
+        wav = vc.run(None, {"mel": mel})[0]  # [1,1,T*256]
+        pieces.append(wav[0, 0])
+    wav = np.concatenate(pieces) if len(pieces) > 1 else pieces[0]
+    wav = wav / (np.abs(wav).max() + 1e-8) * 0.95
+    sf.write(args.output, wav, 24000)
+    print(f"saved: {args.output} ({len(wav)/24000:.2f}s)")
+
+
+if __name__ == "__main__":
+    main()

python/requirements.txt

@@ -0,0 +1,6 @@
+# 推理最小依赖
+onnxruntime>=1.18
+numpy>=1.24
+soundfile>=0.12
+pypinyin>=0.50
+# AX650 NPU 推理（可选）：axengine（AXERA 提供，含 AxEngineExecutionProvider）

python/vocoder_utils.py

@@ -0,0 +1,40 @@
+"""Pretrained mel-conditioned vocoder helpers (NVIDIA BigVGAN)."""
+
+import json
+import os
+
+import torch
+
+from bigvgan.env import AttrDict
+from bigvgan.bigvgan import BigVGAN
+
+VOC_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), "vocoder")
+
+BUILTIN = {
+    "bigvgan_base": ("bigvgan_base_24k_100band.json",
+                     "bigvgan_base_24k_100band.pt"),
+    "bigvgan_v2": ("bigvgan_v2_24k_100band.json",
+                   "bigvgan_v2_24k_100band.pt"),
+}
+
+_cache = {}
+
+
+def load_bigvgan(name="bigvgan_base", device="cpu"):
+    """Load a pretrained BigVGAN generator (frozen, eval mode)."""
+    if name in _cache:
+        return _cache[name]
+    if name not in BUILTIN:
+        raise ValueError(f"Unknown vocoder '{name}', choose from {list(BUILTIN)}")
+    cfg_file, ckpt_file = BUILTIN[name]
+    h = AttrDict(json.load(open(os.path.join(VOC_DIR, cfg_file))))
+    model = BigVGAN(h)
+    ckpt = torch.load(os.path.join(VOC_DIR, ckpt_file), map_location="cpu",
+                      weights_only=True)
+    model.load_state_dict(ckpt["generator"])
+    model.remove_weight_norm()
+    model.eval().to(device)
+    for p in model.parameters():
+        p.requires_grad = False
+    _cache[name] = model
+    return model

reports/EVALUATION.md

@@ -0,0 +1,57 @@
+# 验证报告
+
+## 可懂度（SenseVoice `iic/SenseVoiceSmall`，8 句固定测试）
+
+测试句：
+1. 你好世界，今天天气真不错，适合出去散步。
+2. 机器服务员的生产厂家介绍说这款产品很好用。
+3. 从八月九日十九时到八月十二日上午十时。
+4. 中国人民银行决定下调存款准备金率。
+5. 人工智能技术正在改变我们的生活方式。
+6. 今天中午我想吃一碗牛肉面。
+7. 北京是中国的首都，有着悠久的历史文化。
+8. 请问最近的医院在哪里。
+
+| 配置 | 女声 CER | 男声 CER |
+|---|---|---|
+| CPU 全 fp32（ONNX） | 0.075 | 0.108 |
+
+注：日期句（#3）的系统性偏差为 SenseVoice itn 转写差异（"上午十时"→"上午10"），
+非语音错误。去掉该句后女声 CER 约 0.02。
+
+## 性能（AX650N，10.126.35.115，Ubuntu 22.04）
+
+| 输入 | acoustic(CPU) | vocoder(NPU) | 总耗时 | 有效音频 | RTF |
+|---|---|---|---|---|---|
+| 女声 21 字 | 0.32s | 0.32s | 0.65s | 4.04s | 0.161 |
+| 男声 15 字 | 0.31s | 0.30s | 0.63s | 3.31s | 0.191 |
+| 女声 19 字 | 0.30s | 0.30s | 0.62s | 4.11s | 0.151 |
+| 长句（分块 526 帧） | 0.30s | 0.61s | 0.93s | 5.61s | 0.166 |
+
+## NPU 量化验证结论
+
+### 声学模型（text→mel）：NPU 量化不可行
+
+| 方案 | 结果 |
+|---|---|
+| int8（Percentile/BRecQ） | duration 偏差 13%、mel 渲染 CER~0.9 |
+| U16（激活+权重） | 同上（duration 偏差 13%） |
+| 混合精度 | Pulsar2 编译失败（AxLayerNorm tiling bug） |
+| FP16/F32 | Pulsar2 7.0 不支持 |
+
+根因：VITS 的 flow（仿射耦合）+ duration（对数尺度）对 8/16-bit 量化系统性敏感。
+→ 声学模型必须 CPU fp32（AX650N 实测 0.30s/句）。
+
+### 声码器（mel→wav）：NPU int8 可用
+
+- Pulsar2 7.0 编译，`onnx_opt.enable_onnxsim: true` 必须开启
+  （BigVGAN edge-pad 节点会导致 AxPad shape 错误）
+- 对多数句子可懂（男声真机 CER 0.000）；个别句子有轻微质量波动
+- 板上 CPU fp32 vocoder 实测 >30s/句（rtf>5）不可用 → vocoder 必须 NPU
+
+## 音质优化要点（训练/推理）
+
+- 数据：CosyVoice 官方音色（女）生成 150 条/音色，比原始语料更干净
+- 训练：声码器感知 GAN（mel→冻结 BigVGAN→MPD/MSD）+ 句尾采样 `tail_bias=0.4`
+- 节奏：`length_scale` 女 0.7 / 男 0.85（音节时长贴近自然 180ms）
+- 尾字：`tail_stretch()` 句尾 mel 20→45 帧拉伸（模型句尾音节 duration 偏短）