Initial release: Scrappy Voice Family — four voices in one checkpoint

.gitattributes

@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+samples/clara_intro.wav filter=lfs diff=lfs merge=lfs -text
+samples/pip_intro.wav filter=lfs diff=lfs merge=lfs -text
+samples/scrappy_intro.wav filter=lfs diff=lfs merge=lfs -text
+samples/silas_intro.wav filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1,3 @@
+__pycache__
+*.pyc
+.DS_Store

CITATION.cff

@@ -0,0 +1,11 @@
+cff-version: 1.2.0
+message: If you use this model, please cite it.
+title: 'Inflect-Micro-v2'
+type: software
+authors:
+  - family-names: Song
+    given-names: Owen
+version: 2.0.0
+date-released: 2026-07-24
+url: 'https://huggingface.co/owensong/Inflect-Micro-v2'
+license: Apache-2.0

LICENSE

@@ -0,0 +1,201 @@
+Apache License
+Version 2.0, January 2004
+http://www.apache.org/licenses/
+
+TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+1. Definitions.
+
+"License" shall mean the terms and conditions for use, reproduction,
+and distribution as defined by Sections 1 through 9 of this document.
+
+"Licensor" shall mean the copyright owner or entity authorized by
+the copyright owner that is granting the License.
+
+"Legal Entity" shall mean the union of the acting entity and all
+other entities that control, are controlled by, or are under common
+control with that entity. For the purposes of this definition,
+"control" means (i) the power, direct or indirect, to cause the
+direction or management of such entity, whether by contract or
+otherwise, or (ii) ownership of fifty percent (50%) or more of the
+outstanding shares, or (iii) beneficial ownership of such entity.
+
+"You" (or "Your") shall mean an individual or Legal Entity
+exercising permissions granted by this License.
+
+"Source" form shall mean the preferred form for making modifications,
+including but not limited to software source code, documentation
+source, and configuration files.
+
+"Object" form shall mean any form resulting from mechanical
+transformation or translation of a Source form, including but
+not limited to compiled object code, generated documentation,
+and conversions to other media types.
+
+"Work" shall mean the work of authorship, whether in Source or
+Object form, made available under the License, as indicated by a
+copyright notice that is included in or attached to the work
+(an example is provided in the Appendix below).
+
+"Derivative Works" shall mean any work, whether in Source or Object
+form, that is based on (or derived from) the Work and for which the
+editorial revisions, annotations, elaborations, or other modifications
+represent, as a whole, an original work of authorship. For the purposes
+of this License, Derivative Works shall not include works that remain
+separable from, or merely link (or bind by name) to the interfaces of,
+the Work and Derivative Works thereof.
+
+"Contribution" shall mean any work of authorship, including
+the original version of the Work and any modifications or additions
+to that Work or Derivative Works thereof, that is intentionally
+submitted to Licensor for inclusion in the Work by the copyright owner
+or by an individual or Legal Entity authorized to submit on behalf of
+the copyright owner. For the purposes of this definition, "submitted"
+means any form of electronic, verbal, or written communication sent
+to the Licensor or its representatives, including but not limited to
+communication on electronic mailing lists, source code control systems,
+and issue tracking systems that are managed by, or on behalf of, the
+Licensor for the purpose of discussing and improving the Work, but
+excluding communication that is conspicuously marked or otherwise
+designated in writing by the copyright owner as "Not a Contribution."
+
+"Contributor" shall mean Licensor and any individual or Legal Entity
+on behalf of whom a Contribution has been received by Licensor and
+subsequently incorporated within the Work.
+
+2. Grant of Copyright License. Subject to the terms and conditions of
+this License, each Contributor hereby grants to You a perpetual,
+worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+copyright license to reproduce, prepare Derivative Works of,
+publicly display, publicly perform, sublicense, and distribute the
+Work and such Derivative Works in Source or Object form.
+
+3. Grant of Patent License. Subject to the terms and conditions of
+this License, each Contributor hereby grants to You a perpetual,
+worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+(except as stated in this section) patent license to make, have made,
+use, offer to sell, sell, import, and otherwise transfer the Work,
+where such license applies only to those patent claims licensable
+by such Contributor that are necessarily infringed by their
+Contribution(s) alone or by combination of their Contribution(s)
+with the Work to which such Contribution(s) was submitted. If You
+institute patent litigation against any entity (including a
+cross-claim or counterclaim in a lawsuit) alleging that the Work
+or a Contribution incorporated within the Work constitutes direct
+or contributory patent infringement, then any patent licenses
+granted to You under this License for that Work shall terminate
+as of the date such litigation is filed.
+
+4. Redistribution. You may reproduce and distribute copies of the
+Work or Derivative Works thereof in any medium, with or without
+modifications, and in Source or Object form, provided that You
+meet the following conditions:
+
+(a) You must give any other recipients of the Work or
+Derivative Works a copy of this License; and
+
+(b) You must cause any modified files to carry prominent notices
+stating that You changed the files; and
+
+(c) You must retain, in the Source form of any Derivative Works
+that You distribute, all copyright, patent, trademark, and
+attribution notices from the Source form of the Work,
+excluding those notices that do not pertain to any part of
+the Derivative Works; and
+
+(d) If the Work includes a "NOTICE" text file as part of its
+distribution, then any Derivative Works that You distribute must
+include a readable copy of the attribution notices contained
+within such NOTICE file, excluding those notices that do not
+pertain to any part of the Derivative Works, in at least one
+of the following places: within a NOTICE text file distributed
+as part of the Derivative Works; within the Source form or
+documentation, if provided along with the Derivative Works; or,
+within a display generated by the Derivative Works, if and
+wherever such third-party notices normally appear. The contents
+of the NOTICE file are for informational purposes only and
+do not modify the License. You may add Your own attribution
+notices within Derivative Works that You distribute, alongside
+or as an addendum to the NOTICE text from the Work, provided
+that such additional attribution notices cannot be construed
+as modifying the License.
+
+You may add Your own copyright statement to Your modifications and
+may provide additional or different license terms and conditions
+for use, reproduction, or distribution of Your modifications, or
+for any such Derivative Works as a whole, provided Your use,
+reproduction, and distribution of the Work otherwise complies with
+the conditions stated in this License.
+
+5. Submission of Contributions. Unless You explicitly state otherwise,
+any Contribution intentionally submitted for inclusion in the Work
+by You to the Licensor shall be under the terms and conditions of
+this License, without any additional terms or conditions.
+Notwithstanding the above, nothing herein shall supersede or modify
+the terms of any separate license agreement you may have executed
+with Licensor regarding such Contributions.
+
+6. Trademarks. This License does not grant permission to use the trade
+names, trademarks, service marks, or product names of the Licensor,
+except as required for reasonable and customary use in describing the
+origin of the Work and reproducing the content of the NOTICE file.
+
+7. Disclaimer of Warranty. Unless required by applicable law or
+agreed to in writing, Licensor provides the Work (and each
+Contributor provides its Contributions) on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+implied, including, without limitation, any warranties or conditions
+of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+PARTICULAR PURPOSE. You are solely responsible for determining the
+appropriateness of using or redistributing the Work and assume any
+risks associated with Your exercise of permissions under this License.
+
+8. Limitation of Liability. In no event and under no legal theory,
+whether in tort (including negligence), contract, or otherwise,
+unless required by applicable law (such as deliberate and grossly
+negligent acts) or agreed to in writing, shall any Contributor be
+liable to You for damages, including any direct, indirect, special,
+incidental, or consequential damages of any character arising as a
+result of this License or out of the use or inability to use the
+Work (including but not limited to damages for loss of goodwill,
+work stoppage, computer failure or malfunction, or any and all
+other commercial damages or losses), even if such Contributor
+has been advised of the possibility of such damages.
+
+9. Accepting Warranty or Additional Liability. While redistributing
+the Work or Derivative Works thereof, You may choose to offer,
+and charge a fee for, acceptance of support, warranty, indemnity,
+or other liability obligations and/or rights consistent with this
+License. However, in accepting such obligations, You may act only
+on Your own behalf and on Your sole responsibility, not on behalf
+of any other Contributor, and only if You agree to indemnify,
+defend, and hold each Contributor harmless for any liability
+incurred by, or claims asserted against, such Contributor by reason
+of your accepting any such warranty or additional liability.
+
+END OF TERMS AND CONDITIONS
+
+APPENDIX: How to apply the Apache License to your work.
+
+To apply the Apache License to your work, attach the following
+boilerplate notice, with the fields enclosed by brackets "{}"
+replaced with your own identifying information. (Don't include
+the brackets.) The text should be enclosed in the appropriate
+comment syntax for the file format. We also recommend that a
+file or class name and description of purpose be included on the
+same "printed page" as the copyright notice for easier
+identification within third-party archives.
+
+Copyright {yyyy} {name of copyright owner}
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.

README.md

@@ -0,0 +1,202 @@
+---
+license: apache-2.0
+language: en
+pipeline_tag: text-to-speech
+tags:
+  - text-to-speech
+  - speech-synthesis
+  - vits
+  - multi-speaker
+  - multi-voice
+  - voice-blending
+  - cpu
+  - edge-ai
+  - small-model
+  - 24khz
+  - voice-distillation
+base_model: owensong/Inflect-Micro-v2
+---
+
+# Scrappy Voice Family — four voices in one 10M-param CPU model
+
+Four distinct narrators — **Scrappy**, **Clara**, **Silas**, and **Pip** — sharing a single
+**10.26M-parameter / 41MB** checkpoint that runs **on a plain CPU**, no GPU, no cloud.
+It's a multi-speaker fine-tune of
+[owensong/Inflect-Micro-v2](https://huggingface.co/owensong/Inflect-Micro-v2) (Apache-2.0),
+warm-started from our first voice and trained on all four corpora pooled together.
+
+Listen: [`samples/scrappy_intro.wav`](samples/scrappy_intro.wav) ·
+[`samples/clara_intro.wav`](samples/clara_intro.wav) ·
+[`samples/silas_intro.wav`](samples/silas_intro.wav) ·
+[`samples/pip_intro.wav`](samples/pip_intro.wav) — each voice introducing itself, all four
+generated on a desktop CPU by *this one checkpoint*.
+
+| Voice | id | Character |
+|---|---|---|
+| **Scrappy** | 0 | Warm, expressive narrator — the original ScrappyLabs voice |
+| **Clara** | 1 | Clear professional female narrator, broadcast polish, neutral American with a warm undertone |
+| **Silas** | 2 | Deep, authoritative older male — gravelly, slow, movie-trailer gravitas |
+| **Pip** | 3 | Bright, energetic young American female — quick, playful, podcast-host diction |
+
+**Get it:** [🤗 HuggingFace](https://huggingface.co/scrappylabsai/scrappy-voice-family-1)
+(canonical weights) · Story: [scrappylabs.ai/scrappy-voice](https://scrappylabs.ai/scrappy-voice)
+
+**🎮 Hear the family's first voice live:**
+[interactive demo on Spaces](https://huggingface.co/spaces/scrappylabsai/scrappy-voice-tts)
+— built by the Hugging Face team on free ZeroGPU. Thanks, HF 🤗
+
+## Why a family instead of four models
+
+We shipped [scrappy-voice-1](https://huggingface.co/scrappylabsai/scrappy-voice-1) as a
+single-voice distill, then built three more voices the same way. The obvious next question:
+does one shared model hold four identities without smearing them together?
+
+It does — and pooling the data made it *better*, not worse.
+
+Same recipe as the solo voices, end to end:
+
+1. **Teacher renders each corpus.** A commercial cloud TTS narrator voice generated ~4,400
+   short clips per voice (~5.5h @ 24kHz each) from a text corpus we control — so every
+   transcript is known by construction.
+2. **An ASR gate cleans it.** Every clip is round-tripped through speech recognition and
+   scored against its transcript (≥0.85 word overlap required), alongside signal checks for
+   clipping, silence and duration. Pass rates: Clara 99.4%, Silas 99.3%, Pip 98.5%
+   (Scrappy's original corpus: 98.6%). This is the check that catches audio which says the
+   wrong thing beautifully.
+3. **Warm-start, multi-speaker.** `n_speakers=4`, `gin_channels=256`. Rather than start from
+   the stock base, we warm-started from Scrappy's own 50k checkpoint — chosen by a measured
+   A/B, not vibes: **26.4 vs 28.2 final mel loss at 12k steps**, with ASR intelligibility at
+   parity. An already-distilled voice is a better launch pad than the stock release.
+4. **75,000 steps, batch 24, LR 5e-5, fp32**, over **16,546 pooled training clips**
+   (plus 868 held out) across the four voices.
+
+**Final mel loss 18.4–18.9** — comfortably below the Clara (19.4) and Pip (21.0) solo runs
+and level with Silas's 18.1, the best of the three. Four voices sharing one model landed at
+roughly the quality of the best single-voice run rather than paying for the company, which
+is the pleasant surprise of the whole exercise. Multi-speaker conditioning also costs
+essentially nothing at train time: 4.36 steps/s, same as single-speaker.
+
+Held-out intelligibility on the family model (ASR word-overlap against the intro script):
+Clara **1.00**, Pip **1.00**, Scrappy **0.96**, Silas **0.92** — mean **0.97**. The solo
+models score 1.00 for both Silas and Clara, so a little per-voice sharpness is the price of
+sharing; see [Honest limitations](#honest-limitations).
+
+## Usage
+
+Drop-in for the packaged runtime, with two additions: `voice=` and `blend=`.
+
+```python
+from inference import InflectTTS
+
+tts = InflectTTS(model_dir=".", device="cpu")
+
+print(tts.voices)   # ['scrappy', 'clara', 'silas', 'pip']  (ordered by speaker id)
+
+# pick a voice by name (case-insensitive) or by id
+tts.save("Clear, professional, and easy to listen to.", "clara.wav", voice="clara", seed=7)
+tts.save("Low, slow, and built for weight.",            "silas.wav", voice=2)
+
+# blend two or more voices — weights are normalized for you
+tts.save("Somewhere between the two of them.", "morph.wav",
+         blend={"clara": 0.7, "silas": 0.3}, seed=7)
+
+sample_rate, audio = tts.synthesize("Returns numpy, if you'd rather.", voice="pip")
+```
+
+```bash
+python inference.py --model-dir . --device cpu --voice clara \
+  --text "Hello from the family." --output out.wav
+
+python inference.py --blend "clara=0.7, silas=0.3" \
+  --text "And this is a blend." --output morph.wav
+```
+
+`voice` accepts a name, an int speaker id, or a digit string; omit it and you get voice 0
+(Scrappy). `blend` takes a `{voice: weight}` dict, normalizes the weights to sum to 1, and
+overrides `voice` when both are given. Everything else is unchanged from upstream: English
+only, deterministic seeds, punctuation-aware long-form chunking, `speed` 0.5–2.0,
+`variation` 0.0–1.0. Write numbers out as words for best results.
+
+### Why four voices don't cost four models
+
+Each voice is **one 256-float speaker-embedding row — about 1KB**. The multi-speaker
+machinery (the conditioning pathways through the flow, decoder and posterior encoder) is a
+one-time ~0.9M-parameter cost over the single-voice model; after that, voice number five
+would add roughly a kilobyte, not another 37MB. That's the whole argument for a family:
+you pay for the plumbing once.
+
+Blending works because those embedding rows live in a continuous space. A weighted average
+of two rows is a valid conditioning vector, so `{"clara": 0.7, "silas": 0.3}` renders a voice
+that genuinely sits between them rather than crossfading two renders. Weights are
+normalized, duplicates that resolve to the same voice are summed, and negative,
+non-finite, all-zero or unknown-name inputs are rejected outright.
+
+## Honest limitations
+
+- **Blends near the middle can get uncanny.** Only the four anchor voices were trained. The
+  space between them is interpolation, not supervision — light blends (say 80/20) usually
+  sound like a plausible person; 50/50 blends of very different voices (Silas and Pip, for
+  instance) can land somewhere no human throat goes. Audition before you ship one.
+- **Sharing costs a little per-voice sharpness.** Silas scores 0.92 held-out intelligibility
+  in the family model vs 1.00 solo. If you need one voice at maximum fidelity and don't care
+  about the others, the [solo models](#the-family) still win narrowly.
+- **Prosody is where distillation loses the most.** Timbre and identity transfer well; the
+  teacher's long-range timing instincts — dramatic pauses, phrase-level planning — get
+  averaged. The duration predictor is the smallest organ in a VITS, so expect a flatter read
+  than the source voices.
+- **espeak-ng mispronounces "Silas"** as /siːləz/. Spell it `Sighlus` in synthesis input to
+  get /saɪləs/. A frontend quirk, not a model one — and a good reminder to spot-check how the
+  phonemizer handles proper nouns.
+- Slight texture softness vs. a large vocoder remains at close listening.
+- **English only.** Everything upstream says about language coverage and biases applies.
+- **Not a cloning tool.** These are synthetic personas built from a teacher we're entitled to
+  use. Don't point this pipeline at a real person's voice without their explicit consent.
+
+## The family
+
+| Model | What it is |
+|---|---|
+| [**scrappy-voice-family-1**](https://huggingface.co/scrappylabsai/scrappy-voice-family-1) | ← you are here. All four voices + blending, one checkpoint |
+| [scrappy-voice-1](https://huggingface.co/scrappylabsai/scrappy-voice-1) | Scrappy, solo — the original, and this model's warm start |
+| [clara-voice-1](https://huggingface.co/scrappylabsai/clara-voice-1) | Clara, solo |
+| [silas-voice-1](https://huggingface.co/scrappylabsai/silas-voice-1) | Silas, solo |
+| [pip-voice-1](https://huggingface.co/scrappylabsai/pip-voice-1) | Pip, solo |
+
+Use the family model unless you specifically want one voice at its sharpest, or want the
+smaller single-voice checkpoint.
+
+## Train your own family (`trainer/`)
+
+The `trainer/` directory holds the multi-speaker version of the stack upstream deliberately
+omits:
+
+- `prep_filelists_ms.py` — phonemizes transcripts with the model's own frontend, validates
+  every symbol against the release inventory, and emits speaker-id-tagged filelists plus the
+  `speakers.json` name→id map.
+- `train_ms.py` — the full loop with speaker conditioning: VITS losses, torchaudio mel
+  transforms (slaney/slaney — no librosa dependency), warm-start loading that grows a
+  single-speaker checkpoint into a multi-speaker one, and drop-in candidate export.
+- `eval_candidate_ms.py` — renders fixed prompts from any candidate, per voice, for A/B
+  listening.
+
+You also need the cython monotonic-alignment kernel from the
+[canonical VITS repo](https://github.com/jaywalnut310/vits) (the release stubs it out) —
+build it and drop the package into `runtime/`. Data contract per voice: mono 24kHz clips,
+verified transcripts, 1–5+ hours. **Gate every corpus with ASR round-trips**, and warm-start
+from a voice you've already distilled rather than the stock base — we measured that, and it
+wins.
+
+## Provenance & takedown
+
+All training audio was synthesized by a commercial cloud TTS narrator voice — synthetic
+personas, no real person's voice was cloned. If you're a rights holder with a concern, open
+a discussion on this repo and we'll respond promptly.
+
+## Credits
+
+- **[owensong/Inflect-Micro-v2](https://huggingface.co/owensong/Inflect-Micro-v2)** —
+  base model, runtime, and an unusually honest set of docs (Apache-2.0)
+- [VITS](https://github.com/jaywalnut310/vits) (MIT) — architecture lineage + alignment kernel
+- **The Hugging Face team** — built and gifted us the first demo Space on free ZeroGPU 🤗
+- Built by [ScrappyLabs](https://scrappylabs.ai) — we do this kind of thing to stay sharp.
+  Bring your own AI; we keep it wrangled.

THIRD_PARTY_NOTICES.md

@@ -0,0 +1,36 @@
+# Third-Party Notices
+
+Inflect v2 includes or adapts portions of the following open-source projects.
+Their licenses apply to the corresponding portions; the rest of the Inflect v2
+package is provided under the root Apache-2.0 license.
+
+## VITS
+
+- Project: VITS (`jaywalnut310/vits`)
+- Copyright: Copyright (c) 2021 Jaehyeon Kim
+- License: MIT
+- Packaged license: `third_party/VITS_LICENSE.txt`
+
+The compact model architecture and several inference runtime modules derive from
+VITS. The text frontend also retains its original Keith Ito MIT license at
+`runtime/text/LICENSE`.
+
+## BigVGAN
+
+- Project: BigVGAN (`NVIDIA/BigVGAN`)
+- Copyright: Copyright (c) 2024 NVIDIA CORPORATION
+- License: MIT
+- Packaged license: `third_party/BIGVGAN_LICENSE.txt`
+
+The lightweight alias-free waveform activation implementation derives from
+BigVGAN's alias-free design and was adapted for this compact runtime.
+
+## alias-free-torch
+
+- Project: `alias-free-torch` (`junjun3518/alias-free-torch`)
+- License: Apache License 2.0
+- Packaged license: `third_party/ALIAS_FREE_TORCH_LICENSE.txt`
+
+The anti-aliased activation resampling design used by the compact waveform
+runtime includes concepts and adapted implementation structure from
+`alias-free-torch`.

UPSTREAM-README.md

@@ -0,0 +1,398 @@
+---
+license: apache-2.0
+language:
+  - en
+pipeline_tag: text-to-speech
+model_name: Inflect-Micro-v2
+metrics:
+  - wer
+tags:
+  - text-to-speech
+  - speech-synthesis
+  - local-tts
+  - cpu
+  - edge-ai
+  - small-model
+  - base-model
+  - pytorch
+  - vits
+  - 24khz
+thumbnail: assets/inflect-v2-repository-hero.png
+inference: false
+---
+
+![Inflect-Micro-v2 release cover](assets/inflect-v2-repository-hero.png)
+
+<h1 align="center">Inflect-Micro-v2</h1>
+<p align="center"><strong>Complete local text-to-waveform speech synthesis under 10M parameters.</strong><br>
+Fixed-voice English TTS with deterministic seeds, long-text handling, and CPU or CUDA inference.</p>
+
+> **A note from Owen**
+>
+> Thanks so much for all the support shown on this project! I built and funded Inflect v2 independently - if this release finds a real audience, I would like to continue the project with a broader v3, which might include things like more langauges, voices, and stability improvements. If the model is useful to you, leaving a like on Hugging Face genuinely helps more people discover it.
+
+<p align="center">
+  <a href="https://huggingface.co/spaces/owensong/Inflect-v2"><img alt="Live playground" src="https://img.shields.io/badge/Playground-0B2A54?style=for-the-badge&logo=huggingface&logoColor=white"></a>
+  <a href="https://github.com/owenawsong/Inflect"><img alt="GitHub" src="https://img.shields.io/badge/GitHub-1769E0?style=for-the-badge&logo=github&logoColor=white"></a>
+  <a href="https://huggingface.co/owensong/Inflect-Nano-v2"><img alt="Inflect Nano v2" src="https://img.shields.io/badge/Nano_3.96M-FFFFFF?style=for-the-badge&logo=huggingface&logoColor=1769E0"></a>
+  <a href="https://discord.gg/CVJYedvzvp"><img alt="Inflect Discord" src="https://img.shields.io/badge/Discord-1769E0?style=for-the-badge&logo=discord&logoColor=white"></a>
+  <a href="https://huggingface.co/owensong/Inflect-Micro-v2/blob/main/docs/EVALUATION.md"><img alt="Benchmarks" src="https://img.shields.io/badge/Benchmarks-0B2A54?style=for-the-badge&logo=chartdotjs&logoColor=white"></a>
+</p>
+
+<p align="center"><strong>9,356,513 deployable parameters</strong> · <strong>37.53 MB FP32</strong> · <strong>24 kHz mono output</strong></p>
+
+> **New: public adaptation toolkit**
+>
+> Prepare data, audit train/validation splits, adapt a fixed voice or language, resume training, evaluate checkpoints, and export PyTorch or ONNX packages with the [Inflect adaptation toolkit](https://github.com/owenawsong/Inflect/tree/main/finetune). Adapted quality is experimental and depends on the dataset, frontend, and fluent-speaker evaluation.
+
+---
+
+Inflect v2 uses one public API across two sizes: **Micro prioritizes quality below 10M parameters; Nano prioritizes footprint below 4M.**
+
+<details>
+<summary><strong>Explore this model card</strong></summary>
+
+| Start here | Technical detail |
+| --- | --- |
+| [Listen](#listen) | [Architecture](#architecture-and-parameter-budget) |
+| [Evaluation](#evaluation) | [Controls and long text](#controls-determinism-and-long-text) |
+| [Choose Micro or Nano](#choose-the-right-inflect) | [Data and adaptation](#data-voice-and-adaptation-status) |
+| [Run locally](#run-locally) | [Exports and quantization](https://huggingface.co/owensong/Inflect-Micro-v2/blob/main/docs/EXPORTS.md) |
+| [Adapt a voice or language](https://github.com/owenawsong/Inflect/tree/main/finetune) | [Training and export workflow](https://github.com/owenawsong/Inflect/tree/main/finetune/docs/TRAINING.md) |
+| [Package map](#package-map) | [Evaluation and raw protocol](https://huggingface.co/owensong/Inflect-Micro-v2/blob/main/docs/EVALUATION.md) |
+| [Limitations](#limitations) | [Deployment guide](https://huggingface.co/owensong/Inflect-Micro-v2/blob/main/docs/DEPLOYMENT.md) |
+
+</details>
+
+## Listen
+
+These are held-out text generations, not reconstructions of training audio. Each transcript is shown exactly as passed to the public frontend.
+
+| Test | Exact transcript | Generated audio |
+| --- | --- | --- |
+| **Conversational** | It wasn't until later that I realized what had actually happened. | <audio controls preload="metadata" src="https://huggingface.co/owensong/Inflect-Micro-v2/resolve/main/samples/male/conversational.wav"></audio> |
+| **Punctuation** | First, close the window; second, turn off the lamp; finally, lock the door. | <audio controls preload="metadata" src="https://huggingface.co/owensong/Inflect-Micro-v2/resolve/main/samples/male/punctuation.wav"></audio> |
+| **Numbers** | The package weighs twelve point six kilograms and arrived on July twenty-first. | <audio controls preload="metadata" src="https://huggingface.co/owensong/Inflect-Micro-v2/resolve/main/samples/male/numbers.wav"></audio> |
+| **Names and places** | Gwendolyn photographed the eucalyptus trees outside Ljubljana. | <audio controls preload="metadata" src="https://huggingface.co/owensong/Inflect-Micro-v2/resolve/main/samples/male/names_places.wav"></audio> |
+| **Technical** | The system runs on three core components that all have to stay in sync. | <audio controls preload="metadata" src="https://huggingface.co/owensong/Inflect-Micro-v2/resolve/main/samples/male/technical.wav"></audio> |
+
+## Evaluation
+
+No single metric captures TTS quality. Inflect v2 reports **human preference**, **predicted naturalness**, **multi-ASR intelligibility**, **complete footprint**, and **runtime** separately rather than compressing them into one unverifiable score.
+
+| Community preference ↑ | UTMOS22 ↑ | Two-ASR semantic WER ↓ | Complete FP32 weights ↓ | 4-thread CPU throughput ↑ |
+| ---: | ---: | ---: | ---: | ---: |
+| **66.2%** | **4.395** | **3.99%** | **37.53 MB** | **6.28× real-time** |
+
+The headline row always refers to **Inflect-Micro-v2**. Detailed competitor results and protocol boundaries are kept visible below.
+
+**Comparison set.** Results include [KittenTTS Nano](https://huggingface.co/KittenML/kitten-tts-nano-0.8), [Piper Low](https://huggingface.co/rhasspy/piper-voices), and [Supertonic 3](https://huggingface.co/Supertone/supertonic-3), established compact or local TTS baselines with larger deployable weight footprints than both Inflect releases. Weight sizes are compared at package level, and no single metric is treated as proof of overall superiority.
+
+### 1. Human blind preference
+
+![Community blind listening](assets/evidence/human-preference.svg)
+
+Inflect-Micro-v2 recorded a **66.2% preference rate** (21 wins · 10 losses · 3 ties) in the final anonymous community study. Systems were hidden, left/right order was randomized, and ties count as half a win. This is descriptive community evidence, not formal MOS.
+
+### 2. Predicted naturalness versus footprint
+
+![Predicted quality versus footprint](assets/evidence/quality-vs-footprint.svg)
+
+The UTMOS22 run used 500 identical unseen prompts per voice. KittenTTS and Piper are equal-weight two-voice means; their observed voice ranges appear as whiskers. Supertonic 3-step is reported below the plotted range rather than flattening every other system.
+
+**Inflect-Micro-v2: 4.395 UTMOS22**, 95% bootstrap CI **4.381–4.408**. UTMOS22 is a learned predictor, not human MOS.
+
+### 3. Intelligibility on unseen text
+
+![Two-ASR semantic WER consensus](assets/evidence/asr-consensus.svg)
+
+The headline score is the equal-weight mean of Qwen3-ASR and Nemotron 3.5 corpus WER for **every** system. Whisper is excluded consistently from the headline because it produced insertion-heavy hallucinations on a subset of otherwise intelligible Supertonic 8-step clips. It is not deleted: the complete three-ASR evidence remains below.
+
+<details>
+<summary><strong>Open the complete three-ASR audit</strong></summary>
+
+![Semantic WER across Qwen3-ASR, Nemotron 3.5, and Whisper large-v3](assets/evidence/modern400-three-asr.svg)
+
+| System / voice | Qwen3-ASR ↓ | Nemotron 3.5 ↓ | Whisper large-v3 ↓ |
+| --- | ---: | ---: | ---: |
+| **Inflect-Micro-v2** | **2.52%** | **5.45%** | **2.73%** |
+| **Inflect-Nano-v2** | **2.79%** | **5.63%** | **2.65%** |
+| KittenTTS Nano · Bruno | 2.15% | 3.96% | 2.17% |
+| KittenTTS Nano · Hugo | 2.39% | 3.80% | 2.11% |
+| Piper Low · Danny | 2.62% | 5.60% | 2.55% |
+| Piper Low · Ryan | 2.81% | 5.51% | 2.87% |
+| Supertonic 3 · M2 · 3-step | 3.03% | 6.04% | 3.22% |
+| Supertonic 3 · M2 · 8-step | 2.05% | 3.56% | 8.08% |
+
+For Inflect-Micro-v2, the individual results are **2.52% Qwen3-ASR**, **5.45% Nemotron 3.5**, and **2.73% Whisper large-v3**. The former three-model mean, **3.57%**, is retained only as a descriptive audit value and is not used as the headline score.
+
+</details>
+
+<details>
+<summary><strong>Open evaluator robustness and error-category diagnostics</strong></summary>
+
+![ASR evaluator robustness](assets/evidence/asr-robustness.svg)
+
+![Semantic WER by prompt category](assets/evidence/category-semantic-wer.svg)
+
+These views are diagnostics, not additional leaderboards. They show where the
+recognizers disagree and which prompt categories still produce recoverable
+transcription errors.
+
+</details>
+
+### 4. CPU runtime
+
+Both Inflect releases synthesize comfortably faster than real time on CPU. The
+managed reference run used a Hugging Face **CPU Upgrade** instance (8 vCPU,
+32 GB RAM) with **four framework threads**, end-to-end text-to-waveform timing,
+and 100 fixed Modern400 prompts. Three complete passes were recorded; the first
+cache-building pass was excluded and the table pools passes two and three.
+
+| Release | Steady-state RTF ↓ | Audio / wall time ↑ |
+| --- | ---: | ---: |
+| **Inflect-Micro-v2** | **0.1593** | **6.28×** |
+| **Inflect-Nano-v2** | **0.0933** | **10.72×** |
+
+These are package-level results from the public PyTorch runtime, not a claim
+that Inflect is the fastest compact TTS system. Hardware, frontend behavior,
+framework, compilation, and thread policy all affect small-model measurements.
+
+<details>
+<summary><strong>Open directional compact-system speed context</strong></summary>
+
+The same managed CPU and four-thread policy were used for a shorter comparator
+pass: the identical 50-prompt prefix, repeated twice. KittenTTS and Piper are
+equal-work pooled across their two tested voices.
+
+| System | Audio / wall time ↑ |
+| --- | ---: |
+| Piper Low | 31.37× |
+| KittenTTS Nano | 13.33× |
+| **Inflect-Nano-v2** | **10.72×** |
+| Supertonic 3 · 3-step | 10.15× |
+| **Inflect-Micro-v2** | **6.28×** |
+| Supertonic 3 · 8-step | 4.37× |
+
+Because Inflect uses the larger 100-prompt steady-state run while comparator
+rows use the shorter 50-prompt confirmation pass, this table is deployment
+context rather than a perfectly matched speed leaderboard. Several comparators
+also use optimized ONNX runtimes, while the published Inflect benchmark above
+uses the canonical PyTorch runtime. The separately released Inflect ONNX path
+has not been substituted into those benchmark numbers.
+
+</details>
+
+### 5. Complete weight footprint
+
+![Complete deployable model footprint](assets/evidence/model-footprint.svg)
+
+Voice variants sharing the same weights are merged. Inflect totals include the integrated waveform decoder.
+
+<details>
+<summary><strong>Open the frozen evaluation protocol</strong></summary>
+
+- Modern400 uses 400 identical unseen English prompts per system: 200 fixed modern/stress prompts plus 200 deterministic FLEURS `en_us` test prompts.
+- Exact-text exclusion was checked against 87,362 training transcripts.
+- All ASR inputs are resampled to 16 kHz and scored with the same disclosed English normalizer.
+- UTMOS22 uses `tarepan/SpeechMOS` v1.2.0 on a separate 500-prompt generation set.
+- Headline intervals use 10,000 bootstrap samples.
+- The Modern400 corpus SHA-256 is `b7504ce2dce44a2da82770a6a5dfd2a034fe17e2113980f8a69663ade417a34c`.
+- Prompts, hypotheses, compressed row-level reports, and summaries ship under `evaluation/final/`.
+- Runtime is evaluated separately because framework, thread policy, compilation,
+  and host load can dominate small-model comparisons.
+
+</details>
+
+---
+
+## Choose the right Inflect
+
+| | **Inflect-Nano-v2** | **Inflect-Micro-v2** |
+| --- | ---: | ---: |
+| Complete parameters | 3,966,721 | 9,356,513 |
+| FP32 weights | 15.97 MB | 37.53 MB |
+| Positioning | Smallest practical footprint | Strongest Inflect v2 quality |
+| 24 kHz waveform decoder | Included | Included |
+| Python API and frontend | Same | Same |
+
+**Inflect-Micro-v2** is the quality-focused member of the family. Both models use the same public API and complete text-to-waveform packaging.
+
+## Run locally
+
+### Install
+
+```bash
+python -m pip install --upgrade huggingface_hub
+hf download owensong/Inflect-Micro-v2 --local-dir Inflect-Micro-v2
+cd Inflect-Micro-v2
+python -m pip install -r requirements.txt
+```
+
+This uses the Hub's version-aware downloader and retrieves the complete
+repository. A Git clone also works, but `hf download` is the recommended path
+for ordinary model installation.
+
+### Python
+
+```python
+from inference import InflectTTS
+
+tts = InflectTTS(".", device="cpu")
+tts.save(
+    "A small voice can still have something meaningful to say.",
+    "sample.wav",
+    speed=1.0,
+    variation=0.667,
+    seed=7,
+)
+```
+
+### Download through the Hub
+
+```python
+import sys
+from huggingface_hub import snapshot_download
+
+model_dir = snapshot_download("owensong/Inflect-Micro-v2")
+sys.path.insert(0, model_dir)
+
+from inference import InflectTTS
+
+tts = InflectTTS(model_dir, device="cpu")
+sample_rate, waveform = tts.synthesize("The complete model runs locally.")
+```
+
+The result is a 24 kHz mono `float32` waveform. Long input is split at punctuation-aware boundaries, synthesized chunk by chunk, and joined with controlled pauses.
+
+### ONNX Runtime
+
+The official verified FP32 export is published separately as
+[`Inflect-Micro-v2-ONNX`](https://huggingface.co/owensong/Inflect-Micro-v2-ONNX).
+It supports dynamic lengths, CPU/CUDA/DirectML provider selection,
+deterministic seeds, and the same long-text wrapper without importing PyTorch:
+
+```bash
+git clone https://huggingface.co/owensong/Inflect-Micro-v2-ONNX
+cd Inflect-Micro-v2-ONNX
+python -m pip install -r onnx/requirements.txt
+python onnx/inference_onnx.py \
+  --text "The complete model now runs through ONNX Runtime." \
+  --output sample-onnx.wav \
+  --provider cpu \
+  --seed 7
+```
+
+The neural model is split into `duration.onnx` and `decode.onnx`; together they
+contain the complete learned text-to-waveform path. The English eSpeak-ng
+frontend remains CPU-side code. See the
+[`ONNX repository`](https://huggingface.co/owensong/Inflect-Micro-v2-ONNX)
+for graph contracts, provenance, parity measurements, browser deployment, and
+re-export instructions.
+
+## Release profile
+
+| **Local runtime** | **Long-text handling** |
+| --- | --- |
+| CPU and CUDA inference through the same Python API and CLI. | Punctuation-aware segmentation with controlled pauses and edge fades. |
+| **Repeatable output** | **Auditable evaluation** |
+| Fixed seeds reproduce the same latent sample on the same runtime stack. | Frozen prompts, raw ASR hypotheses, intervals, hashes, and per-system reports are included. |
+
+<details>
+<summary id="architecture-and-parameter-budget"><strong>Architecture and parameter budget</strong></summary>
+
+Inflect v2 is a parameter-efficient VITS-family end-to-end text-to-waveform generator with an English phoneme frontend, monotonic alignment, stochastic latent synthesis, residual coupling flow, and an integrated alias-reduced neural waveform decoder.
+
+| Component | Inflect-Micro-v2 |
+| --- | ---: |
+| Latent channels | 192 |
+| Text hidden channels | 96 |
+| Encoder layers / heads | 3 / 2 |
+| Feed-forward channels | 768 |
+| Flow coupling blocks | 4 |
+| Initial decoder channels | 320 |
+| Upsample rates | 8, 8, 2, 2 |
+| Training segment | 16,384 samples |
+| Output | 24 kHz mono waveform |
+
+The release describes the deployable architecture. Private corpus-construction and optimization details are not part of this open-weight package.
+
+</details>
+
+<details>
+<summary id="controls-determinism-and-long-text"><strong>Controls, determinism, and long text</strong></summary>
+
+| Control | Default | Public range | Meaning |
+| --- | ---: | ---: | --- |
+| `speed` | `1.0` | `0.5–2.0` | Lower is slower; higher is faster. |
+| `variation` | `0.667` | `0.0–1.0` | Lower is steadier; higher samples more latent variation. |
+| `seed` | `0` | integer | Repeats the same stochastic sample on the same runtime stack. |
+
+Long passages are punctuation-aware chunks, not one unlimited autoregressive pass. Chunk boundaries receive short pauses and edge fades. See [`docs/API.md`](https://huggingface.co/owensong/Inflect-Micro-v2/blob/main/docs/API.md) for waveform contracts and concurrency notes.
+
+</details>
+
+<details>
+<summary id="data-voice-and-adaptation-status"><strong>Data, voice, and adaptation status</strong></summary>
+
+The release contains one fixed synthetic English voice. The package does not redistribute a real-speaker recording corpus, does not claim the voice as the identity of a real person, and requires no reference audio or external model at inference.
+
+The base release remains inference-first, but a public **experimental fixed-voice and language adaptation workflow** is now available. A new voice replaces the built-in speaker rather than adding runtime voice cloning. A new language requires owned or licensed speech data, a compatible phoneme frontend, symbol migration, retraining, and fluent-speaker evaluation. Start with the [Inflect adaptation toolkit](https://github.com/owenawsong/Inflect/tree/main/finetune), then review [`docs/DATA_AND_VOICE.md`](https://huggingface.co/owensong/Inflect-Micro-v2/blob/main/docs/DATA_AND_VOICE.md).
+
+</details>
+
+## Package map
+
+| Path | Purpose |
+| --- | --- |
+| `model.pth` | Inference-only generator checkpoint |
+| `config.json` | Architecture and audio configuration; also the Hub download-count query file |
+| `inference.py` | Public Python API and CLI |
+| `inflect_vits_frontend.py` | English normalization, phonemization, and punctuation frontend |
+| `runtime/` | Self-contained model implementation |
+| [`Inflect-Micro-v2-ONNX`](https://huggingface.co/owensong/Inflect-Micro-v2-ONNX) | Separate official FP32 ONNX graphs, torch-free runner, parity report, checksums, and exporter |
+| `samples/` | Held-out example generations |
+| `evaluation/final/` | Frozen benchmark prompts, reports, and protocol artifacts |
+| `docs/` | API, deployment, evaluation, adaptation, and export documentation |
+| `release_manifest.json` | File sizes and SHA-256 hashes |
+
+## Limitations
+
+- English only, with one fixed male voice. This is not zero-shot voice cloning.
+- Unfamiliar phrasing can become flatter, less expressive, or less stable.
+- Numbers, abbreviations, homographs, and uncommon names remain frontend- and context-sensitive.
+- Long passages use punctuation-aware chunking; transitions can differ from a native long-form model pass.
+- Stochastic variation can alter timing and pronunciation. Fix the seed for comparisons.
+- UTMOS22 and ASR scores do not replace controlled human MOS or MUSHRA-style evaluation.
+- Not validated for medical, legal, emergency, or accessibility-critical communication.
+
+## Responsible use
+
+Do not use the included voice to impersonate a real person, deceive listeners, or create fraudulent content. Disclose synthetic speech where the context could otherwise mislead. Users are responsible for applicable laws and the Apache-2.0 license.
+
+## License, integrity, and attribution
+
+Original Inflect code and weights are released under Apache-2.0. Bundled third-party components retain their own notices in [`THIRD_PARTY_NOTICES.md`](https://huggingface.co/owensong/Inflect-Micro-v2/blob/main/THIRD_PARTY_NOTICES.md). `release_manifest.json` records packaged file sizes and SHA-256 hashes.
+
+### Private training scope and contact
+
+Inflect v2 is an **open-weight** release. Deployable weights, inference code, frontend code, evaluation prompts, and release reports are public. The training corpus-generation pipeline, private filtering infrastructure, and full optimization recipe are not part of the public package.
+
+Owen Song may share additional technical context privately for credible research, collaboration, reproducibility, or deployment inquiries when the request has a clear purpose and does not conflict with licensing or data-provenance constraints.
+
+- **Discord:** `b111ue` — fastest for informal technical questions
+- **Community server:** [discord.gg/CVJYedvzvp](https://discord.gg/CVJYedvzvp)
+- **Email:** [owen.aw.song@gmail.com](mailto:owen.aw.song@gmail.com) — preferred for professional inquiries
+
+## Citation
+
+```bibtex
+@software{song2026inflectmicrov2,
+  author = {Owen Song},
+  title = {Inflect-Micro-v2: Complete Local Text-to-Waveform TTS Under 10M Parameters},
+  year = {2026},
+  url = {https://huggingface.co/owensong/Inflect-Micro-v2}
+}
+```
+
+<p align="center"><sub>Designed and developed independently by Owen Song · open weights · Apache-2.0 · complete local text-to-waveform inference</sub></p>

config.json

@@ -0,0 +1,71 @@
+{
+  "format": "inflect_v2_inference_config_v1",
+  "train": {
+    "segment_size": 16384
+  },
+  "data": {
+    "text_cleaners": [],
+    "max_wav_value": 32768.0,
+    "sampling_rate": 24000,
+    "filter_length": 1024,
+    "hop_length": 256,
+    "win_length": 1024,
+    "n_mel_channels": 80,
+    "mel_fmin": 0.0,
+    "mel_fmax": 12000.0,
+    "add_blank": true,
+    "n_speakers": 4,
+    "cleaned_text": true
+  },
+  "model": {
+    "inter_channels": 192,
+    "hidden_channels": 96,
+    "filter_channels": 768,
+    "n_heads": 2,
+    "n_layers": 3,
+    "kernel_size": 3,
+    "p_dropout": 0.1,
+    "resblock": "1",
+    "resblock_kernel_sizes": [
+      3,
+      7,
+      11
+    ],
+    "resblock_dilation_sizes": [
+      [
+        1,
+        3,
+        5
+      ],
+      [
+        1,
+        3,
+        5
+      ],
+      [
+        1,
+        3,
+        5
+      ]
+    ],
+    "upsample_rates": [
+      8,
+      8,
+      2,
+      2
+    ],
+    "upsample_initial_channel": 320,
+    "upsample_kernel_sizes": [
+      16,
+      16,
+      4,
+      4
+    ],
+    "n_layers_q": 3,
+    "use_spectral_norm": false,
+    "use_sdp": false,
+    "inference_only": true,
+    "n_speakers": 4,
+    "gin_channels": 256
+  }
+}

examples/basic.py

@@ -0,0 +1,16 @@
+from pathlib import Path
+import sys
+
+MODEL_DIR = Path(__file__).resolve().parents[1]
+sys.path.insert(0, str(MODEL_DIR))
+
+from inference import InflectTTS
+
+tts = InflectTTS(MODEL_DIR, device="cpu")
+tts.save(
+    "A small local voice can still be useful.",
+    MODEL_DIR / "example.wav",
+    speed=1.0,
+    variation=0.667,
+    seed=7,
+)

examples/long_text.py

@@ -0,0 +1,14 @@
+from pathlib import Path
+import sys
+
+MODEL_DIR = Path(__file__).resolve().parents[1]
+sys.path.insert(0, str(MODEL_DIR))
+
+from inference import InflectTTS
+
+text = (
+    "Long input is divided at punctuation-aware boundaries. "
+    "Each segment is generated locally, then joined with a controlled pause; "
+    "this keeps memory bounded without requiring a remote service."
+)
+InflectTTS(MODEL_DIR, device="cpu").save(text, MODEL_DIR / "long_example.wav", seed=11)

examples/voices.py

@@ -0,0 +1,21 @@
+"""Speak the same line in each voice, then blend two of them."""
+from pathlib import Path
+import sys
+
+MODEL_DIR = Path(__file__).resolve().parents[1]
+sys.path.insert(0, str(MODEL_DIR))
+
+from inference import InflectTTS
+
+engine = InflectTTS(MODEL_DIR, device="cpu")
+line = "The same model, wearing a different voice."
+
+for name in engine.voices:
+    out = MODEL_DIR / f"example_{name}.wav"
+    engine.save(line, out, voice=name)
+    print(f"{name} -> {out.name}")
+
+# Blending interpolates between voices. Weights are normalized.
+out = MODEL_DIR / "example_blend.wav"
+engine.save(line, out, blend={"clara": 0.7, "silas": 0.3})
+print(f"70% clara / 30% silas -> {out.name}")

inference.py

@@ -0,0 +1,362 @@
+from __future__ import annotations
+
+import argparse
+import contextlib
+import io
+import json
+import logging
+import math
+import re
+import sys
+import warnings
+from pathlib import Path
+
+import numpy as np
+import soundfile as sf
+import torch
+
+# CPU thread cap: PyTorch's default (one thread per core) intermittently faults
+# inside oneDNN on many-core machines — measured ~35-40% of runs on a 32-core box.
+# Eight threads is well past this model's scaling knee, so this costs no speed.
+_MAX_THREADS = 8
+try:
+    if torch.get_num_threads() > _MAX_THREADS:
+        torch.set_num_threads(_MAX_THREADS)
+except Exception:  # pragma: no cover - never let a tuning hint break import
+    pass
+
+
+PACKAGE_ROOT = Path(__file__).resolve().parent
+RUNTIME_ROOT = PACKAGE_ROOT / "runtime"
+sys.path.insert(0, str(RUNTIME_ROOT))
+sys.path.insert(0, str(PACKAGE_ROOT))
+
+import commons  # noqa: E402
+import utils  # noqa: E402
+from inflect_vits_frontend import run_vits_frontend  # noqa: E402
+from models import SynthesizerTrn  # noqa: E402
+from text import cleaned_text_to_sequence  # noqa: E402
+from text.symbols import symbols  # noqa: E402
+
+
+SPEAKER_FILE = "speakers.json"
+
+
+def split_text(text: str, limit: int = 280) -> list[str]:
+    normalized = " ".join(text.split())
+    sentences = [
+        part.strip()
+        for part in re.split(r"(?<=[.!?;:])\s+", normalized)
+        if part.strip()
+    ]
+    chunks: list[str] = []
+    for sentence in sentences or [normalized]:
+        while len(sentence) > limit:
+            search = sentence[: limit + 1]
+            punctuation = max(search.rfind(mark) for mark in (",", ";", ":"))
+            split_at = (
+                punctuation + 1
+                if punctuation >= limit // 2
+                else sentence.rfind(" ", 0, limit + 1)
+            )
+            if split_at < limit // 2:
+                split_at = limit
+            chunks.append(sentence[:split_at].strip())
+            sentence = sentence[split_at:].strip()
+        if sentence:
+            chunks.append(sentence)
+    return chunks
+
+
+def boundary_pause_seconds(chunk: str) -> float:
+    ending = chunk.rstrip()[-1:] if chunk.strip() else ""
+    return {
+        "?": 0.28,
+        "!": 0.24,
+        ".": 0.22,
+        ";": 0.16,
+        ":": 0.13,
+        ",": 0.09,
+    }.get(ending, 0.08)
+
+
+def edge_fade(waveform: np.ndarray, sample_rate: int, milliseconds: float = 5.0) -> np.ndarray:
+    frames = min(round(sample_rate * milliseconds / 1000.0), waveform.size // 2)
+    if frames <= 0:
+        return waveform
+    output = waveform.copy()
+    ramp = np.linspace(0.0, 1.0, frames, endpoint=True, dtype=np.float32)
+    output[:frames] *= ramp
+    output[-frames:] *= ramp[::-1]
+    return output
+
+
+def optimize_for_inference(model: SynthesizerTrn) -> None:
+    """Collapse training-time weight normalization without changing outputs."""
+    with contextlib.redirect_stdout(io.StringIO()):
+        model.dec.remove_weight_norm()
+    for flow in model.flow.flows:
+        encoder = getattr(flow, "enc", None)
+        if encoder is not None and hasattr(encoder, "remove_weight_norm"):
+            encoder.remove_weight_norm()
+
+
+def parse_blend(specification: str) -> dict[str, float]:
+    """Read a command line blend such as "clara=0.7,silas=0.3"."""
+    blend: dict[str, float] = {}
+    for part in specification.split(","):
+        entry = part.strip()
+        if not entry:
+            continue
+        name, separator, weight = entry.partition("=")
+        name = name.strip()
+        if not separator or not name:
+            raise ValueError(f"blend entries look like name=weight, got {entry!r}")
+        try:
+            value = float(weight)
+        except ValueError:
+            raise ValueError(f"blend weight for {name!r} is not a number: {weight.strip()!r}") from None
+        blend[name] = blend.get(name, 0.0) + value
+    if not blend:
+        raise ValueError("blend must name at least one voice.")
+    return blend
+
+
+class InflectTTS:
+    def __init__(self, model_dir: str | Path = PACKAGE_ROOT, device: str = "cpu") -> None:
+        self.root = Path(model_dir).resolve()
+        self.device = torch.device(device)
+        self.hps = utils.get_hparams_from_file(str(self.root / "config.json"))
+        self.n_speakers = int(self.hps.model.n_speakers) if "n_speakers" in self.hps.model else 0
+        if self.n_speakers == 1:
+            raise ValueError(
+                "config.json sets n_speakers=1; use 0 for a single voice or 2 or more for a family."
+            )
+        with warnings.catch_warnings():
+            warnings.filterwarnings(
+                "ignore",
+                message="`torch.nn.utils.weight_norm` is deprecated",
+                category=FutureWarning,
+            )
+            self.model = SynthesizerTrn(
+                len(symbols),
+                self.hps.data.filter_length // 2 + 1,
+                self.hps.train.segment_size // self.hps.data.hop_length,
+                **self.hps.model,
+            ).to(self.device).eval()
+        root_logger = logging.getLogger()
+        previous_level = root_logger.level
+        try:
+            root_logger.setLevel(logging.WARNING)
+            utils.load_checkpoint(str(self.root / "model.pth"), self.model, None)
+        finally:
+            root_logger.setLevel(previous_level)
+        self.checkpoint_parameters = sum(parameter.numel() for parameter in self.model.parameters())
+        optimize_for_inference(self.model)
+        self.deployed_parameters = sum(parameter.numel() for parameter in self.model.parameters())
+        self.sample_rate = int(self.hps.data.sampling_rate)
+        self.speakers = self._load_speakers()
+        self.voices = [
+            name for name, _ in sorted(self.speakers.items(), key=lambda item: (item[1], item[0]))
+        ]
+
+    def _load_speakers(self) -> dict[str, int]:
+        """Map voice name to speaker id, falling back to plain integer names."""
+        if self.n_speakers < 2:
+            return {}
+        table: object = {}
+        path = self.root / SPEAKER_FILE
+        if path.is_file():
+            document = json.loads(path.read_text(encoding="utf-8"))
+            if isinstance(document, dict):
+                nested = document.get("speakers")
+                table = nested if isinstance(nested, dict) else document
+        speakers: dict[str, int] = {}
+        if isinstance(table, dict):
+            for name, value in table.items():
+                if isinstance(value, bool) or not isinstance(value, (int, str)):
+                    continue
+                try:
+                    identifier = int(value)
+                except ValueError:
+                    continue
+                if 0 <= identifier < self.n_speakers:
+                    speakers[str(name)] = identifier
+        for identifier in range(self.n_speakers):
+            if identifier not in speakers.values():
+                speakers[str(identifier)] = identifier
+        return speakers
+
+    def voice_id(self, voice: str | int) -> int:
+        """Resolve a voice name (case-insensitive) or a speaker id to a speaker id."""
+        if isinstance(voice, str):
+            wanted = voice.strip().lower()
+            for name, identifier in self.speakers.items():
+                if name.lower() == wanted:
+                    return identifier
+            if wanted.isdigit() and 0 <= int(wanted) < self.n_speakers:
+                return int(wanted)
+            raise ValueError(
+                f"Unknown voice {voice!r}. This model speaks: {', '.join(self.voices)}."
+            )
+        if isinstance(voice, (int, np.integer)) and not isinstance(voice, bool):
+            identifier = int(voice)
+            if 0 <= identifier < self.n_speakers:
+                return identifier
+            raise ValueError(
+                f"Speaker id {identifier} is out of range. This model speaks: {', '.join(self.voices)}."
+            )
+        raise TypeError("voice must be a voice name, a speaker id, or None.")
+
+    def _blend_weights(self, blend: dict[str | int, float]) -> dict[int, float]:
+        if not isinstance(blend, dict):
+            raise TypeError("blend must be a dict of voice name or id to weight.")
+        weights: dict[int, float] = {}
+        for voice, weight in blend.items():
+            value = float(weight)
+            if not math.isfinite(value) or value < 0.0:
+                raise ValueError("blend weights must be finite and not negative.")
+            identifier = self.voice_id(voice)
+            weights[identifier] = weights.get(identifier, 0.0) + value
+        if not weights:
+            raise ValueError("blend must name at least one voice.")
+        total = sum(weights.values())
+        if total <= 0.0:
+            raise ValueError("blend weights must add up to more than zero.")
+        return {identifier: value / total for identifier, value in weights.items()}
+
+    @contextlib.contextmanager
+    def _blended_speaker(self, weights: dict[int, float]):
+        """Stand a one-row speaker table in for emb_g so infer() sees the blend.
+
+        The weighted sum of the speaker embedding rows becomes row 0 of a
+        throwaway table, the model uses it for sid 0, and the original module
+        goes back in the finally block even if synthesis raises. The swap
+        mutates the shared model, so one InflectTTS instance must not blend on
+        two threads at once.
+        """
+        original = self.model.emb_g
+        width = original.embedding_dim
+        vector = torch.zeros(width, dtype=original.weight.dtype, device=original.weight.device)
+        for identifier, weight in weights.items():
+            vector += original.weight[identifier] * weight
+        stand_in = torch.nn.Embedding(
+            1, width, dtype=original.weight.dtype, device=original.weight.device
+        )
+        stand_in.requires_grad_(False)
+        stand_in.weight.copy_(vector.unsqueeze(0))
+        stand_in.eval()
+        self.model.emb_g = stand_in
+        try:
+            yield
+        finally:
+            self.model.emb_g = original
+
+    def _tokens(self, text: str) -> tuple[torch.Tensor, torch.Tensor]:
+        phonemes = run_vits_frontend(text).phoneme_text
+        sequence = cleaned_text_to_sequence(phonemes)
+        if self.hps.data.add_blank:
+            sequence = commons.intersperse(sequence, 0)
+        if not sequence:
+            raise ValueError("The text frontend produced no speakable tokens.")
+        tokens = torch.LongTensor(sequence).to(self.device).unsqueeze(0)
+        lengths = torch.LongTensor([tokens.size(1)]).to(self.device)
+        return tokens, lengths
+
+    @torch.inference_mode()
+    def synthesize(
+        self,
+        text: str,
+        *,
+        voice: str | int | None = None,
+        blend: dict[str | int, float] | None = None,
+        speed: float = 1.0,
+        variation: float = 0.667,
+        seed: int = 0,
+    ) -> tuple[int, np.ndarray]:
+        normalized = " ".join(text.split())
+        if not normalized:
+            raise ValueError("Text must not be empty.")
+        if not 0.5 <= speed <= 2.0:
+            raise ValueError("speed must be between 0.5 and 2.0")
+        if not 0.0 <= variation <= 1.0:
+            raise ValueError("variation must be between 0.0 and 1.0")
+        if self.n_speakers < 2:
+            if voice is not None or blend is not None:
+                raise ValueError("This model has one voice, so voice and blend do not apply.")
+            speaker = None
+            weights = None
+        elif blend is not None:
+            weights = self._blend_weights(blend)
+            speaker = torch.zeros(1, dtype=torch.long, device=self.device)
+        else:
+            weights = None
+            speaker = torch.LongTensor([self.voice_id(0 if voice is None else voice)]).to(self.device)
+        chunks = split_text(normalized)
+        pieces: list[np.ndarray] = []
+        speaking = self._blended_speaker(weights) if weights else contextlib.nullcontext()
+        with speaking:
+            for index, chunk in enumerate(chunks):
+                if index:
+                    pieces.append(
+                        np.zeros(
+                            round(self.sample_rate * boundary_pause_seconds(chunks[index - 1])),
+                            dtype=np.float32,
+                        )
+                    )
+                tokens, lengths = self._tokens(chunk)
+                torch.manual_seed(seed + index)
+                if self.device.type == "cuda":
+                    torch.cuda.manual_seed_all(seed + index)
+                waveform = self.model.infer(
+                    tokens,
+                    lengths,
+                    sid=speaker,
+                    noise_scale=variation,
+                    noise_scale_w=0.8,
+                    length_scale=1.0 / speed,
+                    max_len=4000,
+                )[0][0, 0].float().cpu().numpy()
+                pieces.append(edge_fade(waveform, self.sample_rate))
+        waveform = np.clip(np.concatenate(pieces), -1.0, 1.0)
+        return self.sample_rate, waveform
+
+    def save(self, text: str, output: str | Path, **kwargs: object) -> Path:
+        destination = Path(output)
+        destination.parent.mkdir(parents=True, exist_ok=True)
+        sample_rate, waveform = self.synthesize(text, **kwargs)
+        sf.write(destination, waveform, sample_rate)
+        return destination
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description="Run standalone Inflect v2 synthesis.")
+    parser.add_argument("--model-dir", type=Path, default=PACKAGE_ROOT)
+    parser.add_argument("--text", required=True)
+    parser.add_argument("--output", type=Path, required=True)
+    parser.add_argument("--device", default="cpu")
+    parser.add_argument("--voice", default=None, help="voice name or speaker id, e.g. clara")
+    parser.add_argument(
+        "--blend",
+        default=None,
+        help='morph between voices, e.g. "clara=0.7,silas=0.3" (overrides --voice)',
+    )
+    parser.add_argument("--speed", type=float, default=1.0)
+    parser.add_argument("--variation", type=float, default=0.667)
+    parser.add_argument("--seed", type=int, default=0)
+    args = parser.parse_args()
+    engine = InflectTTS(args.model_dir, args.device)
+    engine.save(
+        args.text,
+        args.output,
+        voice=args.voice,
+        blend=parse_blend(args.blend) if args.blend else None,
+        speed=args.speed,
+        variation=args.variation,
+        seed=args.seed,
+    )
+    print(f"wrote {args.output} at {engine.sample_rate} Hz")
+
+
+if __name__ == "__main__":
+    main()

inflect_nano_v2_frontend.py

@@ -0,0 +1,395 @@
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import re
+import sys
+from dataclasses import asdict, dataclass
+from datetime import date
+from pathlib import Path
+
+from num2words import num2words
+
+
+MONTHS = [
+    "January",
+    "February",
+    "March",
+    "April",
+    "May",
+    "June",
+    "July",
+    "August",
+    "September",
+    "October",
+    "November",
+    "December",
+]
+
+WORD_OVERRIDES = {
+    "Qwen3": "Qwen three",
+    "Qwen": "Qwen",
+    "PyTorch": "pie torch",
+    "SQLite": "ess cue lite",
+    "USB-C": "you ess bee see",
+    "RTX 3060": "ar tee ex thirty sixty",
+    "RTX 3090": "ar tee ex thirty ninety",
+    "RTX 4090": "ar tee ex forty ninety",
+    "RTX 5080": "ar tee ex fifty eighty",
+    "RTX 5090": "ar tee ex fifty ninety",
+}
+
+LETTER_NAMES = {
+    "A": "ay",
+    "B": "bee",
+    "C": "see",
+    "D": "dee",
+    "E": "ee",
+    "F": "eff",
+    "G": "gee",
+    "H": "aitch",
+    "I": "eye",
+    "J": "jay",
+    "K": "kay",
+    "L": "ell",
+    "M": "em",
+    "N": "en",
+    "O": "oh",
+    "P": "pee",
+    "Q": "cue",
+    "R": "ar",
+    "S": "ess",
+    "T": "tee",
+    "U": "you",
+    "V": "vee",
+    "W": "double you",
+    "X": "ex",
+    "Y": "why",
+    "Z": "zee",
+}
+
+ABBREVIATIONS = {
+    "Dr.": "doctor",
+    "Mr.": "mister",
+    "Mrs.": "missus",
+    "Ms.": "miss",
+    "Prof.": "professor",
+    "St.": "saint",
+    "vs.": "versus",
+    "etc.": "et cetera",
+    "e.g.": "for example",
+    "i.e.": "that is",
+}
+
+PUNCT_TRANSLATION = str.maketrans(
+    {
+        "\u2018": "'",
+        "\u2019": "'",
+        "\u201c": '"',
+        "\u201d": '"',
+        "\u2013": "-",
+        "\u2014": ", ",
+        "\u2026": "...",
+        "(": ", ",
+        ")": ", ",
+        "[": ", ",
+        "]": ", ",
+        "{": ", ",
+        "}": ", ",
+    }
+)
+
+_ESPEAK_CONFIGURED = False
+_ESPEAK_BACKEND = None
+
+
+@dataclass
+class FrontendOutput:
+    raw_text: str
+    normalized_text: str
+    phoneme_text: str
+    tokens: list[str]
+    token_count: int
+
+
+def _words(value: int | float, *, ordinal: bool = False) -> str:
+    if ordinal:
+        text = num2words(value, to="ordinal")
+    else:
+        text = num2words(value)
+    return text.replace("-", " ").replace(",", "")
+
+
+def _digit_words(text: str) -> str:
+    return " ".join(_words(int(ch)) for ch in text if ch.isdigit())
+
+
+def _identifier_digits(text: str) -> str:
+    words = []
+    for index, character in enumerate(text):
+        if not character.isdigit():
+            continue
+        words.append("oh" if character == "0" and index > 0 else _words(int(character)))
+    return " ".join(words)
+
+
+def _expand_identifier_token(token: str) -> str:
+    match = re.fullmatch(r"([A-Za-z]?)(\d+)([A-Za-z]?)", token)
+    if match is None:
+        return token
+    prefix, digits, suffix = match.groups()
+    pieces = []
+    if prefix:
+        pieces.append(LETTER_NAMES[prefix.upper()])
+    if len(digits) == 3 or digits.startswith("0"):
+        pieces.append(_identifier_digits(digits))
+    else:
+        pieces.append(_words(int(digits)))
+    if suffix:
+        pieces.append(LETTER_NAMES[suffix.upper()])
+    return " ".join(pieces)
+
+
+def _expand_labeled_identifier(match: re.Match[str]) -> str:
+    return f"{match.group(1)} {_expand_identifier_token(match.group(2))}"
+
+
+def _expand_street_number(match: re.Match[str]) -> str:
+    return _identifier_digits(match.group(1))
+
+
+def _expand_money(match: re.Match[str]) -> str:
+    raw = match.group(1).replace(",", "")
+    dollars, _, cents = raw.partition(".")
+    dollar_count = int(dollars)
+    parts = [_words(dollar_count), "dollar" if dollar_count == 1 else "dollars"]
+    if cents:
+        cents = cents[:2].ljust(2, "0")
+        cent_count = int(cents)
+        if cent_count:
+            parts.extend(["and", _words(cent_count), "cent" if cent_count == 1 else "cents"])
+    return " ".join(parts)
+
+
+def _expand_date_slash(match: re.Match[str]) -> str:
+    month = int(match.group(1))
+    day = int(match.group(2))
+    year = int(match.group(3))
+    try:
+        date(year, month, day)
+    except ValueError:
+        return match.group(0)
+    return f"{MONTHS[month - 1]} {_words(day, ordinal=True)} {_words(year)}"
+
+
+def _expand_time(match: re.Match[str]) -> str:
+    hour = int(match.group(1))
+    minute = int(match.group(2))
+    suffix = match.group(3) or ""
+    pieces = [_words(hour)]
+    if minute == 0:
+        pieces.append("o clock")
+    elif minute < 10:
+        pieces.extend(["oh", _words(minute)])
+    else:
+        pieces.append(_words(minute))
+    if suffix:
+        suffix = suffix.lower().replace(".", "")
+        pieces.extend(list(suffix))
+    return " ".join(pieces)
+
+
+def _expand_bare_hour_time(match: re.Match[str]) -> str:
+    hour = int(match.group(1))
+    suffix = re.sub(r"[^A-Za-z]", "", match.group(2)).lower()
+    return f"{_words(hour)} {' '.join(suffix)}"
+
+
+def _expand_version(match: re.Match[str]) -> str:
+    return " point ".join(_words(int(part)) for part in match.group(0).split("."))
+
+
+def _expand_decimal(match: re.Match[str]) -> str:
+    whole, frac = match.group(1), match.group(2)
+    return f"{_words(int(whole))} point {_digit_words(frac)}"
+
+
+def _expand_ordinal(match: re.Match[str]) -> str:
+    return _words(int(match.group(1)), ordinal=True)
+
+
+def _expand_number(match: re.Match[str]) -> str:
+    value = match.group(0).replace(",", "")
+    if len(value) >= 5 and not value.startswith("20"):
+        return _digit_words(value)
+    return _words(int(value))
+
+
+def _expand_phone(match: re.Match[str]) -> str:
+    left, right = match.group(1), match.group(2)
+    return f"{_digit_words(left)}, {_digit_words(right)}"
+
+
+def _expand_acronym(match: re.Match[str]) -> str:
+    acronym = match.group(0)
+    if len(acronym) <= 1:
+        return acronym
+    return " ".join(LETTER_NAMES.get(ch, ch) for ch in acronym)
+
+
+def normalize_text(text: str) -> str:
+    text = text.translate(PUNCT_TRANSLATION)
+    text = re.sub(r"\s+", " ", text).strip()
+
+    for src, dst in WORD_OVERRIDES.items():
+        text = re.sub(rf"\b{re.escape(src)}\b", dst, text)
+    for src, dst in ABBREVIATIONS.items():
+        text = re.sub(rf"\b{re.escape(src)}", dst, text, flags=re.IGNORECASE)
+
+    text = re.sub(r"\b([A-Z])(?:\.([A-Z]))+\.", lambda m: " ".join(re.findall(r"[A-Z]", m.group(0))), text)
+    text = re.sub(
+        r"\b(apartment|apt\.?|suite|unit|room|flight|extension|order|invoice|locker|aisle|gate)\s+([A-Za-z]?\d{1,4}[A-Za-z]?)\b",
+        _expand_labeled_identifier,
+        text,
+        flags=re.IGNORECASE,
+    )
+    text = re.sub(
+        r"\b(\d{3})(?=\s+(?:North|South|East|West)\b)",
+        _expand_street_number,
+        text,
+        flags=re.IGNORECASE,
+    )
+    text = re.sub(r"\$(\d[\d,]*(?:\.\d{1,2})?)", _expand_money, text)
+    text = re.sub(r"\b(0?[1-9]|1[0-2])/(0?[1-9]|[12]\d|3[01])/(20\d{2}|19\d{2})\b", _expand_date_slash, text)
+    text = re.sub(r"\b(\d{1,2}):(\d{2})\s*([AaPp]\.?\s*[Mm]\.?)?\b", _expand_time, text)
+    text = re.sub(r"\b(\d{1,2})\s*([AaPp]\.?\s*[Mm]\.?)\b", _expand_bare_hour_time, text)
+    text = re.sub(r"\b(\d{3})-(\d{4})\b", _expand_phone, text)
+    text = re.sub(r"\b\d+(?:\.\d+){2,}\b", _expand_version, text)
+    text = re.sub(r"\b(\d+)\.(\d+)\b", _expand_decimal, text)
+    text = re.sub(r"\b(\d+)(st|nd|rd|th)\b", _expand_ordinal, text, flags=re.IGNORECASE)
+    text = re.sub(r"\b\d[\d,]*\b", _expand_number, text)
+    text = re.sub(r"\b[A-Z]{2,}\b", _expand_acronym, text)
+    text = re.sub(r",(?:\s*,)+", ",", text)
+    text = re.sub(r",\s*([.!?])", r"\1", text)
+    text = re.sub(r"\s+([,;:.!?])", r"\1", text)
+    text = re.sub(r"([,;:.!?])(?=\S)", r"\1 ", text)
+    return re.sub(r"\s+", " ", text).strip()
+
+
+def _configure_espeak() -> None:
+    global _ESPEAK_CONFIGURED
+    if _ESPEAK_CONFIGURED:
+        return
+
+    # Prefer the persistent distro library for long Linux preprocessing jobs.
+    # espeakng-loader extracts a temporary shared object, which can exhaust mmap
+    # resources when phonemizer repeatedly creates backends over a large corpus.
+    system_libraries = (
+        Path("/usr/lib/x86_64-linux-gnu/libespeak-ng.so.1"),
+        Path("/usr/lib/aarch64-linux-gnu/libespeak-ng.so.1"),
+        Path("/usr/lib64/libespeak-ng.so.1"),
+    )
+    system_library = next((path for path in system_libraries if path.is_file()), None)
+    if system_library is not None:
+        os.environ.setdefault("PHONEMIZER_ESPEAK_LIBRARY", str(system_library))
+    else:
+        import espeakng_loader
+
+        os.environ.setdefault("PHONEMIZER_ESPEAK_LIBRARY", espeakng_loader.get_library_path())
+        os.environ.setdefault("ESPEAK_DATA_PATH", espeakng_loader.get_data_path())
+        espeakng_loader.make_library_available()
+        espeakng_loader.load_library()
+    _ESPEAK_CONFIGURED = True
+
+
+def phonemize_normalized_text(normalized_text: str) -> str:
+    global _ESPEAK_BACKEND
+    _configure_espeak()
+    from phonemizer.backend import EspeakBackend
+    from phonemizer.separator import Separator
+
+    if _ESPEAK_BACKEND is None:
+        _ESPEAK_BACKEND = EspeakBackend(
+            language="en-us",
+            preserve_punctuation=True,
+            with_stress=True,
+            language_switch="remove-flags",
+        )
+    return _ESPEAK_BACKEND.phonemize(
+        [normalized_text],
+        separator=Separator(phone=" ", word=" | ", syllable=""),
+        strip=True,
+        njobs=1,
+    )[0]
+
+
+def tokenize_phoneme_text(phoneme_text: str) -> list[str]:
+    text = phoneme_text.replace("|", " <word> ")
+    text = re.sub(r"([,;:.!?])", r" \1 ", text)
+    tokens = [tok for tok in re.split(r"\s+", text.strip()) if tok]
+    return tokens
+
+
+def run_frontend(text: str) -> FrontendOutput:
+    normalized = normalize_text(text)
+    phoneme_text = phonemize_normalized_text(normalized)
+    tokens = tokenize_phoneme_text(phoneme_text)
+    return FrontendOutput(
+        raw_text=text,
+        normalized_text=normalized,
+        phoneme_text=phoneme_text,
+        tokens=tokens,
+        token_count=len(tokens),
+    )
+
+
+def _iter_input_rows(path: Path):
+    with path.open("r", encoding="utf-8") as f:
+        for line_number, line in enumerate(f, 1):
+            line = line.strip()
+            if not line:
+                continue
+            if line.startswith("{"):
+                row = json.loads(line)
+                text = row.get("target_text") or row.get("text") or row.get("source_text")
+                if not text:
+                    raise ValueError(f"No text field found at {path}:{line_number}")
+                yield row, str(text)
+            else:
+                yield {"line_number": line_number}, line
+
+
+def main() -> None:
+    if hasattr(sys.stdout, "reconfigure"):
+        sys.stdout.reconfigure(encoding="utf-8", errors="replace")
+    if hasattr(sys.stderr, "reconfigure"):
+        sys.stderr.reconfigure(encoding="utf-8", errors="replace")
+
+    ap = argparse.ArgumentParser(description="Inflect-Nano-v2 English text frontend.")
+    ap.add_argument("--text", help="Single text string to normalize and phonemize.")
+    ap.add_argument("--input", type=Path, help="Text file or JSONL to process.")
+    ap.add_argument("--out", type=Path, help="Output JSONL path for --input.")
+    args = ap.parse_args()
+
+    if bool(args.text) == bool(args.input):
+        raise SystemExit("Provide exactly one of --text or --input.")
+
+    if args.text:
+        print(json.dumps(asdict(run_frontend(args.text)), ensure_ascii=False, indent=2))
+        return
+
+    if not args.out:
+        raise SystemExit("--out is required with --input.")
+
+    args.out.parent.mkdir(parents=True, exist_ok=True)
+    with args.out.open("w", encoding="utf-8") as f:
+        for metadata, text in _iter_input_rows(args.input):
+            result = asdict(run_frontend(text))
+            result["metadata"] = metadata
+            f.write(json.dumps(result, ensure_ascii=False) + "\n")
+    print(f"Wrote {args.out}")
+
+
+if __name__ == "__main__":
+    main()

inflect_vits_frontend.py

@@ -0,0 +1,67 @@
+from __future__ import annotations
+
+import re
+from dataclasses import dataclass
+
+from inflect_nano_v2_frontend import _configure_espeak, normalize_text
+
+
+# eSpeak is the general fallback. This table contains verified exceptions only;
+# every entry is covered by a regression test and listening audit.
+PHONEME_OVERRIDES = {
+    "sˈæskɐtʃˌuːən": "sɐskˈætʃəwən",
+    "flʊɹɹˈɛsənt": "flʊˈɹɛsənt",
+}
+
+
+@dataclass(frozen=True)
+class VitsFrontendOutput:
+    raw_text: str
+    normalized_text: str
+    phoneme_text: str
+
+
+def phonemize_normalized(normalized_text: str) -> str:
+    return phonemize_normalized_batch([normalized_text], jobs=1)[0]
+
+
+def _apply_phoneme_overrides(phoneme_text: str) -> str:
+    for source, replacement in PHONEME_OVERRIDES.items():
+        phoneme_text = phoneme_text.replace(source, replacement)
+    return re.sub(r"\s+", " ", phoneme_text).strip()
+
+
+def phonemize_normalized_batch(normalized_texts: list[str], *, jobs: int = 1) -> list[str]:
+    if not normalized_texts:
+        return []
+    _configure_espeak()
+    from phonemizer import phonemize
+
+    phoneme_texts = phonemize(
+        normalized_texts,
+        language="en-us",
+        backend="espeak",
+        strip=True,
+        preserve_punctuation=True,
+        with_stress=True,
+        njobs=jobs,
+    )
+    return [_apply_phoneme_overrides(text) for text in phoneme_texts]
+
+
+def run_vits_frontend_batch(texts: list[str], *, jobs: int = 1) -> list[VitsFrontendOutput]:
+    normalized = [normalize_text(text) for text in texts]
+    phonemes = phonemize_normalized_batch(normalized, jobs=jobs)
+    return [
+        VitsFrontendOutput(raw_text=raw, normalized_text=norm, phoneme_text=phones)
+        for raw, norm, phones in zip(texts, normalized, phonemes, strict=True)
+    ]
+
+
+def run_vits_frontend(text: str) -> VitsFrontendOutput:
+    normalized = normalize_text(text)
+    return VitsFrontendOutput(
+        raw_text=text,
+        normalized_text=normalized,
+        phoneme_text=phonemize_normalized(normalized),
+    )

model.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e6cd2fb4940cad8c6bba0d4cc04ac9cefca7f012e22e54e657c29011c16cf0a7
+size 41136135

requirements-tested.txt

@@ -0,0 +1,12 @@
+# Exact package versions used to validate this release.
+# Install the CPU or CUDA PyTorch wheel appropriate for your platform first.
+torch==2.13.0
+numpy==2.5.1
+scipy==1.18.0
+soundfile==0.14.0
+phonemizer==3.3.0
+num2words==0.5.14
+Unidecode==1.4.0
+# phonemizer needs an espeak-ng backend. This release was validated against a
+# system espeak-ng install; the espeakng-loader wheel works too if you prefer
+# not to install one system-wide.

requirements.txt

@@ -0,0 +1,8 @@
+torch>=2.6
+numpy>=1.26,<3
+scipy>=1.13
+soundfile>=0.13
+phonemizer>=3.3
+espeakng-loader>=0.2.4
+num2words>=0.5.14
+Unidecode>=1.3.8

runtime/attentions.py

@@ -0,0 +1,303 @@
+import copy
+import math
+import numpy as np
+import torch
+from torch import nn
+from torch.nn import functional as F
+
+import commons
+import modules
+from modules import LayerNorm
+   
+
+class Encoder(nn.Module):
+  def __init__(self, hidden_channels, filter_channels, n_heads, n_layers, kernel_size=1, p_dropout=0., window_size=4, **kwargs):
+    super().__init__()
+    self.hidden_channels = hidden_channels
+    self.filter_channels = filter_channels
+    self.n_heads = n_heads
+    self.n_layers = n_layers
+    self.kernel_size = kernel_size
+    self.p_dropout = p_dropout
+    self.window_size = window_size
+
+    self.drop = nn.Dropout(p_dropout)
+    self.attn_layers = nn.ModuleList()
+    self.norm_layers_1 = nn.ModuleList()
+    self.ffn_layers = nn.ModuleList()
+    self.norm_layers_2 = nn.ModuleList()
+    for i in range(self.n_layers):
+      self.attn_layers.append(MultiHeadAttention(hidden_channels, hidden_channels, n_heads, p_dropout=p_dropout, window_size=window_size))
+      self.norm_layers_1.append(LayerNorm(hidden_channels))
+      self.ffn_layers.append(FFN(hidden_channels, hidden_channels, filter_channels, kernel_size, p_dropout=p_dropout))
+      self.norm_layers_2.append(LayerNorm(hidden_channels))
+
+  def forward(self, x, x_mask):
+    attn_mask = x_mask.unsqueeze(2) * x_mask.unsqueeze(-1)
+    x = x * x_mask
+    for i in range(self.n_layers):
+      y = self.attn_layers[i](x, x, attn_mask)
+      y = self.drop(y)
+      x = self.norm_layers_1[i](x + y)
+
+      y = self.ffn_layers[i](x, x_mask)
+      y = self.drop(y)
+      x = self.norm_layers_2[i](x + y)
+    x = x * x_mask
+    return x
+
+
+class Decoder(nn.Module):
+  def __init__(self, hidden_channels, filter_channels, n_heads, n_layers, kernel_size=1, p_dropout=0., proximal_bias=False, proximal_init=True, **kwargs):
+    super().__init__()
+    self.hidden_channels = hidden_channels
+    self.filter_channels = filter_channels
+    self.n_heads = n_heads
+    self.n_layers = n_layers
+    self.kernel_size = kernel_size
+    self.p_dropout = p_dropout
+    self.proximal_bias = proximal_bias
+    self.proximal_init = proximal_init
+
+    self.drop = nn.Dropout(p_dropout)
+    self.self_attn_layers = nn.ModuleList()
+    self.norm_layers_0 = nn.ModuleList()
+    self.encdec_attn_layers = nn.ModuleList()
+    self.norm_layers_1 = nn.ModuleList()
+    self.ffn_layers = nn.ModuleList()
+    self.norm_layers_2 = nn.ModuleList()
+    for i in range(self.n_layers):
+      self.self_attn_layers.append(MultiHeadAttention(hidden_channels, hidden_channels, n_heads, p_dropout=p_dropout, proximal_bias=proximal_bias, proximal_init=proximal_init))
+      self.norm_layers_0.append(LayerNorm(hidden_channels))
+      self.encdec_attn_layers.append(MultiHeadAttention(hidden_channels, hidden_channels, n_heads, p_dropout=p_dropout))
+      self.norm_layers_1.append(LayerNorm(hidden_channels))
+      self.ffn_layers.append(FFN(hidden_channels, hidden_channels, filter_channels, kernel_size, p_dropout=p_dropout, causal=True))
+      self.norm_layers_2.append(LayerNorm(hidden_channels))
+
+  def forward(self, x, x_mask, h, h_mask):
+    """
+    x: decoder input
+    h: encoder output
+    """
+    self_attn_mask = commons.subsequent_mask(x_mask.size(2)).to(device=x.device, dtype=x.dtype)
+    encdec_attn_mask = h_mask.unsqueeze(2) * x_mask.unsqueeze(-1)
+    x = x * x_mask
+    for i in range(self.n_layers):
+      y = self.self_attn_layers[i](x, x, self_attn_mask)
+      y = self.drop(y)
+      x = self.norm_layers_0[i](x + y)
+
+      y = self.encdec_attn_layers[i](x, h, encdec_attn_mask)
+      y = self.drop(y)
+      x = self.norm_layers_1[i](x + y)
+      
+      y = self.ffn_layers[i](x, x_mask)
+      y = self.drop(y)
+      x = self.norm_layers_2[i](x + y)
+    x = x * x_mask
+    return x
+
+
+class MultiHeadAttention(nn.Module):
+  def __init__(self, channels, out_channels, n_heads, p_dropout=0., window_size=None, heads_share=True, block_length=None, proximal_bias=False, proximal_init=False):
+    super().__init__()
+    assert channels % n_heads == 0
+
+    self.channels = channels
+    self.out_channels = out_channels
+    self.n_heads = n_heads
+    self.p_dropout = p_dropout
+    self.window_size = window_size
+    self.heads_share = heads_share
+    self.block_length = block_length
+    self.proximal_bias = proximal_bias
+    self.proximal_init = proximal_init
+    self.attn = None
+
+    self.k_channels = channels // n_heads
+    self.conv_q = nn.Conv1d(channels, channels, 1)
+    self.conv_k = nn.Conv1d(channels, channels, 1)
+    self.conv_v = nn.Conv1d(channels, channels, 1)
+    self.conv_o = nn.Conv1d(channels, out_channels, 1)
+    self.drop = nn.Dropout(p_dropout)
+
+    if window_size is not None:
+      n_heads_rel = 1 if heads_share else n_heads
+      rel_stddev = self.k_channels**-0.5
+      self.emb_rel_k = nn.Parameter(torch.randn(n_heads_rel, window_size * 2 + 1, self.k_channels) * rel_stddev)
+      self.emb_rel_v = nn.Parameter(torch.randn(n_heads_rel, window_size * 2 + 1, self.k_channels) * rel_stddev)
+
+    nn.init.xavier_uniform_(self.conv_q.weight)
+    nn.init.xavier_uniform_(self.conv_k.weight)
+    nn.init.xavier_uniform_(self.conv_v.weight)
+    if proximal_init:
+      with torch.no_grad():
+        self.conv_k.weight.copy_(self.conv_q.weight)
+        self.conv_k.bias.copy_(self.conv_q.bias)
+      
+  def forward(self, x, c, attn_mask=None):
+    q = self.conv_q(x)
+    k = self.conv_k(c)
+    v = self.conv_v(c)
+    
+    x, self.attn = self.attention(q, k, v, mask=attn_mask)
+
+    x = self.conv_o(x)
+    return x
+
+  def attention(self, query, key, value, mask=None):
+    # reshape [b, d, t] -> [b, n_h, t, d_k]
+    b, d, t_s, t_t = (*key.size(), query.size(2))
+    query = query.view(b, self.n_heads, self.k_channels, t_t).transpose(2, 3)
+    key = key.view(b, self.n_heads, self.k_channels, t_s).transpose(2, 3)
+    value = value.view(b, self.n_heads, self.k_channels, t_s).transpose(2, 3)
+
+    scores = torch.matmul(query / math.sqrt(self.k_channels), key.transpose(-2, -1))
+    if self.window_size is not None:
+      assert t_s == t_t, "Relative attention is only available for self-attention."
+      key_relative_embeddings = self._get_relative_embeddings(self.emb_rel_k, t_s)
+      rel_logits = self._matmul_with_relative_keys(query /math.sqrt(self.k_channels), key_relative_embeddings)
+      scores_local = self._relative_position_to_absolute_position(rel_logits)
+      scores = scores + scores_local
+    if self.proximal_bias:
+      assert t_s == t_t, "Proximal bias is only available for self-attention."
+      scores = scores + self._attention_bias_proximal(t_s).to(device=scores.device, dtype=scores.dtype)
+    if mask is not None:
+      scores = scores.masked_fill(mask == 0, -1e4)
+      if self.block_length is not None:
+        assert t_s == t_t, "Local attention is only available for self-attention."
+        block_mask = torch.ones_like(scores).triu(-self.block_length).tril(self.block_length)
+        scores = scores.masked_fill(block_mask == 0, -1e4)
+    p_attn = F.softmax(scores, dim=-1) # [b, n_h, t_t, t_s]
+    p_attn = self.drop(p_attn)
+    output = torch.matmul(p_attn, value)
+    if self.window_size is not None:
+      relative_weights = self._absolute_position_to_relative_position(p_attn)
+      value_relative_embeddings = self._get_relative_embeddings(self.emb_rel_v, t_s)
+      output = output + self._matmul_with_relative_values(relative_weights, value_relative_embeddings)
+    output = output.transpose(2, 3).contiguous().view(b, d, t_t) # [b, n_h, t_t, d_k] -> [b, d, t_t]
+    return output, p_attn
+
+  def _matmul_with_relative_values(self, x, y):
+    """
+    x: [b, h, l, m]
+    y: [h or 1, m, d]
+    ret: [b, h, l, d]
+    """
+    ret = torch.matmul(x, y.unsqueeze(0))
+    return ret
+
+  def _matmul_with_relative_keys(self, x, y):
+    """
+    x: [b, h, l, d]
+    y: [h or 1, m, d]
+    ret: [b, h, l, m]
+    """
+    ret = torch.matmul(x, y.unsqueeze(0).transpose(-2, -1))
+    return ret
+
+  def _get_relative_embeddings(self, relative_embeddings, length):
+    max_relative_position = 2 * self.window_size + 1
+    # Pad first before slice to avoid using cond ops.
+    pad_length = max(length - (self.window_size + 1), 0)
+    slice_start_position = max((self.window_size + 1) - length, 0)
+    slice_end_position = slice_start_position + 2 * length - 1
+    if pad_length > 0:
+      padded_relative_embeddings = F.pad(
+          relative_embeddings,
+          commons.convert_pad_shape([[0, 0], [pad_length, pad_length], [0, 0]]))
+    else:
+      padded_relative_embeddings = relative_embeddings
+    used_relative_embeddings = padded_relative_embeddings[:,slice_start_position:slice_end_position]
+    return used_relative_embeddings
+
+  def _relative_position_to_absolute_position(self, x):
+    """
+    x: [b, h, l, 2*l-1]
+    ret: [b, h, l, l]
+    """
+    batch, heads, length, _ = x.size()
+    # Concat columns of pad to shift from relative to absolute indexing.
+    x = F.pad(x, commons.convert_pad_shape([[0,0],[0,0],[0,0],[0,1]]))
+
+    # Concat extra elements so to add up to shape (len+1, 2*len-1).
+    x_flat = x.view([batch, heads, length * 2 * length])
+    x_flat = F.pad(x_flat, commons.convert_pad_shape([[0,0],[0,0],[0,length-1]]))
+
+    # Reshape and slice out the padded elements.
+    x_final = x_flat.view([batch, heads, length+1, 2*length-1])[:, :, :length, length-1:]
+    return x_final
+
+  def _absolute_position_to_relative_position(self, x):
+    """
+    x: [b, h, l, l]
+    ret: [b, h, l, 2*l-1]
+    """
+    batch, heads, length, _ = x.size()
+    # padd along column
+    x = F.pad(x, commons.convert_pad_shape([[0, 0], [0, 0], [0, 0], [0, length-1]]))
+    x_flat = x.view([batch, heads, length**2 + length*(length -1)])
+    # add 0's in the beginning that will skew the elements after reshape
+    x_flat = F.pad(x_flat, commons.convert_pad_shape([[0, 0], [0, 0], [length, 0]]))
+    x_final = x_flat.view([batch, heads, length, 2*length])[:,:,:,1:]
+    return x_final
+
+  def _attention_bias_proximal(self, length):
+    """Bias for self-attention to encourage attention to close positions.
+    Args:
+      length: an integer scalar.
+    Returns:
+      a Tensor with shape [1, 1, length, length]
+    """
+    r = torch.arange(length, dtype=torch.float32)
+    diff = torch.unsqueeze(r, 0) - torch.unsqueeze(r, 1)
+    return torch.unsqueeze(torch.unsqueeze(-torch.log1p(torch.abs(diff)), 0), 0)
+
+
+class FFN(nn.Module):
+  def __init__(self, in_channels, out_channels, filter_channels, kernel_size, p_dropout=0., activation=None, causal=False):
+    super().__init__()
+    self.in_channels = in_channels
+    self.out_channels = out_channels
+    self.filter_channels = filter_channels
+    self.kernel_size = kernel_size
+    self.p_dropout = p_dropout
+    self.activation = activation
+    self.causal = causal
+
+    if causal:
+      self.padding = self._causal_padding
+    else:
+      self.padding = self._same_padding
+
+    self.conv_1 = nn.Conv1d(in_channels, filter_channels, kernel_size)
+    self.conv_2 = nn.Conv1d(filter_channels, out_channels, kernel_size)
+    self.drop = nn.Dropout(p_dropout)
+
+  def forward(self, x, x_mask):
+    x = self.conv_1(self.padding(x * x_mask))
+    if self.activation == "gelu":
+      x = x * torch.sigmoid(1.702 * x)
+    else:
+      x = torch.relu(x)
+    x = self.drop(x)
+    x = self.conv_2(self.padding(x * x_mask))
+    return x * x_mask
+  
+  def _causal_padding(self, x):
+    if self.kernel_size == 1:
+      return x
+    pad_l = self.kernel_size - 1
+    pad_r = 0
+    padding = [[0, 0], [0, 0], [pad_l, pad_r]]
+    x = F.pad(x, commons.convert_pad_shape(padding))
+    return x
+
+  def _same_padding(self, x):
+    if self.kernel_size == 1:
+      return x
+    pad_l = (self.kernel_size - 1) // 2
+    pad_r = self.kernel_size // 2
+    padding = [[0, 0], [0, 0], [pad_l, pad_r]]
+    x = F.pad(x, commons.convert_pad_shape(padding))
+    return x

runtime/commons.py

@@ -0,0 +1,161 @@
+import math
+import numpy as np
+import torch
+from torch import nn
+from torch.nn import functional as F
+
+
+def init_weights(m, mean=0.0, std=0.01):
+  classname = m.__class__.__name__
+  if classname.find("Conv") != -1:
+    m.weight.data.normal_(mean, std)
+
+
+def get_padding(kernel_size, dilation=1):
+  return int((kernel_size*dilation - dilation)/2)
+
+
+def convert_pad_shape(pad_shape):
+  l = pad_shape[::-1]
+  pad_shape = [item for sublist in l for item in sublist]
+  return pad_shape
+
+
+def intersperse(lst, item):
+  result = [item] * (len(lst) * 2 + 1)
+  result[1::2] = lst
+  return result
+
+
+def kl_divergence(m_p, logs_p, m_q, logs_q):
+  """KL(P||Q)"""
+  kl = (logs_q - logs_p) - 0.5
+  kl += 0.5 * (torch.exp(2. * logs_p) + ((m_p - m_q)**2)) * torch.exp(-2. * logs_q)
+  return kl
+
+
+def rand_gumbel(shape):
+  """Sample from the Gumbel distribution, protect from overflows."""
+  uniform_samples = torch.rand(shape) * 0.99998 + 0.00001
+  return -torch.log(-torch.log(uniform_samples))
+
+
+def rand_gumbel_like(x):
+  g = rand_gumbel(x.size()).to(dtype=x.dtype, device=x.device)
+  return g
+
+
+def slice_segments(x, ids_str, segment_size=4):
+  ret = torch.zeros_like(x[:, :, :segment_size])
+  for i in range(x.size(0)):
+    idx_str = ids_str[i]
+    idx_end = idx_str + segment_size
+    ret[i] = x[i, :, idx_str:idx_end]
+  return ret
+
+
+def rand_slice_segments(x, x_lengths=None, segment_size=4):
+  b, d, t = x.size()
+  if x_lengths is None:
+    x_lengths = t
+  ids_str_max = x_lengths - segment_size + 1
+  ids_str = (torch.rand([b]).to(device=x.device) * ids_str_max).to(dtype=torch.long)
+  ret = slice_segments(x, ids_str, segment_size)
+  return ret, ids_str
+
+
+def get_timing_signal_1d(
+    length, channels, min_timescale=1.0, max_timescale=1.0e4):
+  position = torch.arange(length, dtype=torch.float)
+  num_timescales = channels // 2
+  log_timescale_increment = (
+      math.log(float(max_timescale) / float(min_timescale)) /
+      (num_timescales - 1))
+  inv_timescales = min_timescale * torch.exp(
+      torch.arange(num_timescales, dtype=torch.float) * -log_timescale_increment)
+  scaled_time = position.unsqueeze(0) * inv_timescales.unsqueeze(1)
+  signal = torch.cat([torch.sin(scaled_time), torch.cos(scaled_time)], 0)
+  signal = F.pad(signal, [0, 0, 0, channels % 2])
+  signal = signal.view(1, channels, length)
+  return signal
+
+
+def add_timing_signal_1d(x, min_timescale=1.0, max_timescale=1.0e4):
+  b, channels, length = x.size()
+  signal = get_timing_signal_1d(length, channels, min_timescale, max_timescale)
+  return x + signal.to(dtype=x.dtype, device=x.device)
+
+
+def cat_timing_signal_1d(x, min_timescale=1.0, max_timescale=1.0e4, axis=1):
+  b, channels, length = x.size()
+  signal = get_timing_signal_1d(length, channels, min_timescale, max_timescale)
+  return torch.cat([x, signal.to(dtype=x.dtype, device=x.device)], axis)
+
+
+def subsequent_mask(length):
+  mask = torch.tril(torch.ones(length, length)).unsqueeze(0).unsqueeze(0)
+  return mask
+
+
+@torch.jit.script
+def fused_add_tanh_sigmoid_multiply(input_a, input_b, n_channels):
+  n_channels_int = n_channels[0]
+  in_act = input_a + input_b
+  t_act = torch.tanh(in_act[:, :n_channels_int, :])
+  s_act = torch.sigmoid(in_act[:, n_channels_int:, :])
+  acts = t_act * s_act
+  return acts
+
+
+def convert_pad_shape(pad_shape):
+  l = pad_shape[::-1]
+  pad_shape = [item for sublist in l for item in sublist]
+  return pad_shape
+
+
+def shift_1d(x):
+  x = F.pad(x, convert_pad_shape([[0, 0], [0, 0], [1, 0]]))[:, :, :-1]
+  return x
+
+
+def sequence_mask(length, max_length=None):
+  if max_length is None:
+    max_length = length.max()
+  x = torch.arange(max_length, dtype=length.dtype, device=length.device)
+  return x.unsqueeze(0) < length.unsqueeze(1)
+
+
+def generate_path(duration, mask):
+  """
+  duration: [b, 1, t_x]
+  mask: [b, 1, t_y, t_x]
+  """
+  device = duration.device
+  
+  b, _, t_y, t_x = mask.shape
+  cum_duration = torch.cumsum(duration, -1)
+  
+  cum_duration_flat = cum_duration.view(b * t_x)
+  path = sequence_mask(cum_duration_flat, t_y).to(mask.dtype)
+  path = path.view(b, t_x, t_y)
+  path = path - F.pad(path, convert_pad_shape([[0, 0], [1, 0], [0, 0]]))[:, :-1]
+  path = path.unsqueeze(1).transpose(2,3) * mask
+  return path
+
+
+def clip_grad_value_(parameters, clip_value, norm_type=2):
+  if isinstance(parameters, torch.Tensor):
+    parameters = [parameters]
+  parameters = list(filter(lambda p: p.grad is not None, parameters))
+  norm_type = float(norm_type)
+  if clip_value is not None:
+    clip_value = float(clip_value)
+
+  total_norm = 0
+  for p in parameters:
+    param_norm = p.grad.data.norm(norm_type)
+    total_norm += param_norm.item() ** norm_type
+    if clip_value is not None:
+      p.grad.data.clamp_(min=-clip_value, max=clip_value)
+  total_norm = total_norm ** (1. / norm_type)
+  return total_norm

runtime/inflect_alias_free.py

@@ -0,0 +1,143 @@
+"""Lightweight alias-free waveform blocks derived from NVIDIA BigVGAN.
+
+BigVGAN and alias-free-torch are MIT/Apache-2.0 licensed. The implementation
+is kept local so Inflect can train without BigVGAN's optional CUDA extension.
+"""
+
+import math
+
+import torch
+from torch import nn
+from torch.nn import functional as F
+from torch.nn.utils import remove_weight_norm, weight_norm
+
+from commons import get_padding, init_weights
+
+
+def kaiser_sinc_filter1d(cutoff: float, half_width: float, kernel_size: int):
+  even = kernel_size % 2 == 0
+  half_size = kernel_size // 2
+  delta_f = 4 * half_width
+  attenuation = 2.285 * (half_size - 1) * math.pi * delta_f + 7.95
+  if attenuation > 50.0:
+    beta = 0.1102 * (attenuation - 8.7)
+  elif attenuation >= 21.0:
+    beta = 0.5842 * (attenuation - 21) ** 0.4 + 0.07886 * (attenuation - 21.0)
+  else:
+    beta = 0.0
+  window = torch.kaiser_window(kernel_size, beta=beta, periodic=False)
+  if even:
+    time = torch.arange(-half_size, half_size) + 0.5
+  else:
+    time = torch.arange(kernel_size) - half_size
+  values = 2 * cutoff * window * torch.sinc(2 * cutoff * time)
+  values /= values.sum()
+  return values.view(1, 1, kernel_size)
+
+
+class UpSample1d(nn.Module):
+  def __init__(self, ratio=2, kernel_size=12):
+    super().__init__()
+    self.ratio = ratio
+    self.stride = ratio
+    self.kernel_size = kernel_size
+    self.pad = kernel_size // ratio - 1
+    self.pad_left = self.pad * ratio + (kernel_size - ratio) // 2
+    self.pad_right = self.pad * ratio + (kernel_size - ratio + 1) // 2
+    self.register_buffer(
+        "filter",
+        kaiser_sinc_filter1d(0.5 / ratio, 0.6 / ratio, kernel_size))
+
+  def forward(self, x):
+    channels = x.shape[1]
+    x = F.pad(x, (self.pad, self.pad), mode="replicate")
+    x = self.ratio * F.conv_transpose1d(
+        x, self.filter.expand(channels, -1, -1),
+        stride=self.stride, groups=channels)
+    return x[..., self.pad_left:-self.pad_right]
+
+
+class DownSample1d(nn.Module):
+  def __init__(self, ratio=2, kernel_size=12):
+    super().__init__()
+    self.ratio = ratio
+    self.kernel_size = kernel_size
+    self.pad_left = kernel_size // 2 - int(kernel_size % 2 == 0)
+    self.pad_right = kernel_size // 2
+    self.register_buffer(
+        "filter",
+        kaiser_sinc_filter1d(0.5 / ratio, 0.6 / ratio, kernel_size))
+
+  def forward(self, x):
+    channels = x.shape[1]
+    x = F.pad(x, (self.pad_left, self.pad_right), mode="replicate")
+    return F.conv1d(
+        x, self.filter.expand(channels, -1, -1),
+        stride=self.ratio, groups=channels)
+
+
+class SnakeBeta(nn.Module):
+  def __init__(self, channels: int, logscale: bool = True):
+    super().__init__()
+    initial = torch.zeros(channels) if logscale else torch.ones(channels)
+    self.alpha = nn.Parameter(initial.clone())
+    self.beta = nn.Parameter(initial.clone())
+    self.logscale = logscale
+
+  def forward(self, x):
+    alpha = self.alpha.view(1, -1, 1)
+    beta = self.beta.view(1, -1, 1)
+    if self.logscale:
+      alpha = alpha.exp()
+      beta = beta.exp()
+    return x + torch.sin(x * alpha).square() / (beta + 1e-9)
+
+
+class AliasFreeActivation1d(nn.Module):
+  def __init__(self, activation: nn.Module):
+    super().__init__()
+    self.upsample = UpSample1d()
+    self.act = activation
+    self.downsample = DownSample1d()
+
+  def forward(self, x):
+    return self.downsample(self.act(self.upsample(x)))
+
+
+class AliasFreeResBlock1(nn.Module):
+  """Shape-compatible VITS ResBlock1 with filtered SnakeBeta activations."""
+
+  def __init__(self, channels, kernel_size=3, dilation=(1, 3, 5), logscale=True):
+    super().__init__()
+    self.convs1 = nn.ModuleList([
+        weight_norm(nn.Conv1d(
+            channels, channels, kernel_size, 1,
+            dilation=d, padding=get_padding(kernel_size, d)))
+        for d in dilation
+    ])
+    self.convs2 = nn.ModuleList([
+        weight_norm(nn.Conv1d(
+            channels, channels, kernel_size, 1,
+            dilation=1, padding=get_padding(kernel_size, 1)))
+        for _ in dilation
+    ])
+    self.convs1.apply(init_weights)
+    self.convs2.apply(init_weights)
+    self.activations = nn.ModuleList([
+        AliasFreeActivation1d(SnakeBeta(channels, logscale=logscale))
+        for _ in range(2 * len(dilation))
+    ])
+
+  def forward(self, x, x_mask=None):
+    first = self.activations[::2]
+    second = self.activations[1::2]
+    for conv1, conv2, act1, act2 in zip(self.convs1, self.convs2, first, second):
+      residual = conv2(act2(conv1(act1(x))))
+      x = x + residual
+    return x
+
+  def remove_weight_norm(self):
+    for layer in self.convs1:
+      remove_weight_norm(layer)
+    for layer in self.convs2:
+      remove_weight_norm(layer)

runtime/models.py

@@ -0,0 +1,571 @@
+import copy
+import math
+import torch
+from torch import nn
+from torch.nn import functional as F
+
+import commons
+import modules
+from inflect_alias_free import AliasFreeActivation1d, AliasFreeResBlock1, SnakeBeta
+import attentions
+import monotonic_align
+
+from torch.nn import Conv1d, ConvTranspose1d, AvgPool1d, Conv2d
+from torch.nn.utils import weight_norm, remove_weight_norm, spectral_norm
+from commons import init_weights, get_padding
+
+
+class StochasticDurationPredictor(nn.Module):
+  def __init__(self, in_channels, filter_channels, kernel_size, p_dropout, n_flows=4, gin_channels=0):
+    super().__init__()
+    filter_channels = in_channels # it needs to be removed from future version.
+    self.in_channels = in_channels
+    self.filter_channels = filter_channels
+    self.kernel_size = kernel_size
+    self.p_dropout = p_dropout
+    self.n_flows = n_flows
+    self.gin_channels = gin_channels
+
+    self.log_flow = modules.Log()
+    self.flows = nn.ModuleList()
+    self.flows.append(modules.ElementwiseAffine(2))
+    for i in range(n_flows):
+      self.flows.append(modules.ConvFlow(2, filter_channels, kernel_size, n_layers=3))
+      self.flows.append(modules.Flip())
+
+    self.post_pre = nn.Conv1d(1, filter_channels, 1)
+    self.post_proj = nn.Conv1d(filter_channels, filter_channels, 1)
+    self.post_convs = modules.DDSConv(filter_channels, kernel_size, n_layers=3, p_dropout=p_dropout)
+    self.post_flows = nn.ModuleList()
+    self.post_flows.append(modules.ElementwiseAffine(2))
+    for i in range(4):
+      self.post_flows.append(modules.ConvFlow(2, filter_channels, kernel_size, n_layers=3))
+      self.post_flows.append(modules.Flip())
+
+    self.pre = nn.Conv1d(in_channels, filter_channels, 1)
+    self.proj = nn.Conv1d(filter_channels, filter_channels, 1)
+    self.convs = modules.DDSConv(filter_channels, kernel_size, n_layers=3, p_dropout=p_dropout)
+    if gin_channels != 0:
+      self.cond = nn.Conv1d(gin_channels, filter_channels, 1)
+
+  def forward(self, x, x_mask, w=None, g=None, reverse=False, noise_scale=1.0):
+    x = torch.detach(x)
+    x = self.pre(x)
+    if g is not None:
+      g = torch.detach(g)
+      x = x + self.cond(g)
+    x = self.convs(x, x_mask)
+    x = self.proj(x) * x_mask
+
+    if not reverse:
+      flows = self.flows
+      assert w is not None
+
+      logdet_tot_q = 0 
+      h_w = self.post_pre(w)
+      h_w = self.post_convs(h_w, x_mask)
+      h_w = self.post_proj(h_w) * x_mask
+      e_q = torch.randn(w.size(0), 2, w.size(2)).to(device=x.device, dtype=x.dtype) * x_mask
+      z_q = e_q
+      for flow in self.post_flows:
+        z_q, logdet_q = flow(z_q, x_mask, g=(x + h_w))
+        logdet_tot_q += logdet_q
+      z_u, z1 = torch.split(z_q, [1, 1], 1) 
+      u = torch.sigmoid(z_u) * x_mask
+      z0 = (w - u) * x_mask
+      logdet_tot_q += torch.sum((F.logsigmoid(z_u) + F.logsigmoid(-z_u)) * x_mask, [1,2])
+      logq = torch.sum(-0.5 * (math.log(2*math.pi) + (e_q**2)) * x_mask, [1,2]) - logdet_tot_q
+
+      logdet_tot = 0
+      z0, logdet = self.log_flow(z0, x_mask)
+      logdet_tot += logdet
+      z = torch.cat([z0, z1], 1)
+      for flow in flows:
+        z, logdet = flow(z, x_mask, g=x, reverse=reverse)
+        logdet_tot = logdet_tot + logdet
+      nll = torch.sum(0.5 * (math.log(2*math.pi) + (z**2)) * x_mask, [1,2]) - logdet_tot
+      return nll + logq # [b]
+    else:
+      flows = list(reversed(self.flows))
+      flows = flows[:-2] + [flows[-1]] # remove a useless vflow
+      z = torch.randn(x.size(0), 2, x.size(2)).to(device=x.device, dtype=x.dtype) * noise_scale
+      for flow in flows:
+        z = flow(z, x_mask, g=x, reverse=reverse)
+      z0, z1 = torch.split(z, [1, 1], 1)
+      logw = z0
+      return logw
+
+
+class DurationPredictor(nn.Module):
+  def __init__(self, in_channels, filter_channels, kernel_size, p_dropout, gin_channels=0):
+    super().__init__()
+
+    self.in_channels = in_channels
+    self.filter_channels = filter_channels
+    self.kernel_size = kernel_size
+    self.p_dropout = p_dropout
+    self.gin_channels = gin_channels
+
+    self.drop = nn.Dropout(p_dropout)
+    self.conv_1 = nn.Conv1d(in_channels, filter_channels, kernel_size, padding=kernel_size//2)
+    self.norm_1 = modules.LayerNorm(filter_channels)
+    self.conv_2 = nn.Conv1d(filter_channels, filter_channels, kernel_size, padding=kernel_size//2)
+    self.norm_2 = modules.LayerNorm(filter_channels)
+    self.proj = nn.Conv1d(filter_channels, 1, 1)
+
+    if gin_channels != 0:
+      self.cond = nn.Conv1d(gin_channels, in_channels, 1)
+
+  def forward(self, x, x_mask, g=None):
+    x = torch.detach(x)
+    if g is not None:
+      g = torch.detach(g)
+      x = x + self.cond(g)
+    x = self.conv_1(x * x_mask)
+    x = torch.relu(x)
+    x = self.norm_1(x)
+    x = self.drop(x)
+    x = self.conv_2(x * x_mask)
+    x = torch.relu(x)
+    x = self.norm_2(x)
+    x = self.drop(x)
+    x = self.proj(x * x_mask)
+    return x * x_mask
+
+
+class TextEncoder(nn.Module):
+  def __init__(self,
+      n_vocab,
+      out_channels,
+      hidden_channels,
+      filter_channels,
+      n_heads,
+      n_layers,
+      kernel_size,
+      p_dropout):
+    super().__init__()
+    self.n_vocab = n_vocab
+    self.out_channels = out_channels
+    self.hidden_channels = hidden_channels
+    self.filter_channels = filter_channels
+    self.n_heads = n_heads
+    self.n_layers = n_layers
+    self.kernel_size = kernel_size
+    self.p_dropout = p_dropout
+
+    self.emb = nn.Embedding(n_vocab, hidden_channels)
+    nn.init.normal_(self.emb.weight, 0.0, hidden_channels**-0.5)
+
+    self.encoder = attentions.Encoder(
+      hidden_channels,
+      filter_channels,
+      n_heads,
+      n_layers,
+      kernel_size,
+      p_dropout)
+    self.proj= nn.Conv1d(hidden_channels, out_channels * 2, 1)
+
+  def forward(self, x, x_lengths):
+    x = self.emb(x) * math.sqrt(self.hidden_channels) # [b, t, h]
+    x = torch.transpose(x, 1, -1) # [b, h, t]
+    x_mask = torch.unsqueeze(commons.sequence_mask(x_lengths, x.size(2)), 1).to(x.dtype)
+
+    x = self.encoder(x * x_mask, x_mask)
+    stats = self.proj(x) * x_mask
+
+    m, logs = torch.split(stats, self.out_channels, dim=1)
+    return x, m, logs, x_mask
+
+
+class ResidualCouplingBlock(nn.Module):
+  def __init__(self,
+      channels,
+      hidden_channels,
+      kernel_size,
+      dilation_rate,
+      n_layers,
+      n_flows=4,
+      gin_channels=0):
+    super().__init__()
+    self.channels = channels
+    self.hidden_channels = hidden_channels
+    self.kernel_size = kernel_size
+    self.dilation_rate = dilation_rate
+    self.n_layers = n_layers
+    self.n_flows = n_flows
+    self.gin_channels = gin_channels
+
+    self.flows = nn.ModuleList()
+    for i in range(n_flows):
+      self.flows.append(modules.ResidualCouplingLayer(channels, hidden_channels, kernel_size, dilation_rate, n_layers, gin_channels=gin_channels, mean_only=True))
+      self.flows.append(modules.Flip())
+
+  def forward(self, x, x_mask, g=None, reverse=False):
+    if not reverse:
+      for flow in self.flows:
+        x, _ = flow(x, x_mask, g=g, reverse=reverse)
+    else:
+      for flow in reversed(self.flows):
+        x = flow(x, x_mask, g=g, reverse=reverse)
+    return x
+
+
+class PosteriorEncoder(nn.Module):
+  def __init__(self,
+      in_channels,
+      out_channels,
+      hidden_channels,
+      kernel_size,
+      dilation_rate,
+      n_layers,
+      gin_channels=0):
+    super().__init__()
+    self.in_channels = in_channels
+    self.out_channels = out_channels
+    self.hidden_channels = hidden_channels
+    self.kernel_size = kernel_size
+    self.dilation_rate = dilation_rate
+    self.n_layers = n_layers
+    self.gin_channels = gin_channels
+
+    self.pre = nn.Conv1d(in_channels, hidden_channels, 1)
+    self.enc = modules.WN(hidden_channels, kernel_size, dilation_rate, n_layers, gin_channels=gin_channels)
+    self.proj = nn.Conv1d(hidden_channels, out_channels * 2, 1)
+
+  def forward(self, x, x_lengths, g=None):
+    x_mask = torch.unsqueeze(commons.sequence_mask(x_lengths, x.size(2)), 1).to(x.dtype)
+    x = self.pre(x) * x_mask
+    x = self.enc(x, x_mask, g=g)
+    stats = self.proj(x) * x_mask
+    m, logs = torch.split(stats, self.out_channels, dim=1)
+    z = (m + torch.randn_like(m) * torch.exp(logs)) * x_mask
+    return z, m, logs, x_mask
+
+
+class Generator(torch.nn.Module):
+    def __init__(self, initial_channel, resblock, resblock_kernel_sizes, resblock_dilation_sizes, upsample_rates, upsample_initial_channel, upsample_kernel_sizes, gin_channels=0, decoder_alias_free=False, decoder_alias_free_start_stage=2, decoder_snake_logscale=True):
+        super(Generator, self).__init__()
+        self.num_kernels = len(resblock_kernel_sizes)
+        self.num_upsamples = len(upsample_rates)
+        self.conv_pre = Conv1d(initial_channel, upsample_initial_channel, 7, 1, padding=3)
+        resblock_class = modules.ResBlock1 if resblock == '1' else modules.ResBlock2
+        self.decoder_alias_free = bool(decoder_alias_free)
+        self.decoder_alias_free_start_stage = int(decoder_alias_free_start_stage)
+
+        self.ups = nn.ModuleList()
+        for i, (u, k) in enumerate(zip(upsample_rates, upsample_kernel_sizes)):
+            self.ups.append(weight_norm(
+                ConvTranspose1d(upsample_initial_channel//(2**i), upsample_initial_channel//(2**(i+1)),
+                                k, u, padding=(k-u)//2)))
+
+        self.resblocks = nn.ModuleList()
+        for i in range(len(self.ups)):
+            ch = upsample_initial_channel//(2**(i+1))
+            for j, (k, d) in enumerate(zip(resblock_kernel_sizes, resblock_dilation_sizes)):
+                if self.decoder_alias_free and i >= self.decoder_alias_free_start_stage:
+                    self.resblocks.append(AliasFreeResBlock1(
+                        ch, k, d, logscale=decoder_snake_logscale))
+                else:
+                    self.resblocks.append(resblock_class(ch, k, d))
+
+        self.alias_free_pre_activations = nn.ModuleList()
+        if self.decoder_alias_free:
+            for i in range(self.num_upsamples):
+                channels = upsample_initial_channel // (2 ** i)
+                if i >= self.decoder_alias_free_start_stage:
+                    self.alias_free_pre_activations.append(
+                        AliasFreeActivation1d(nn.LeakyReLU(modules.LRELU_SLOPE)))
+                else:
+                    self.alias_free_pre_activations.append(nn.Identity())
+            self.alias_free_post_activation = AliasFreeActivation1d(
+                SnakeBeta(ch, logscale=decoder_snake_logscale))
+
+        self.conv_post = Conv1d(ch, 1, 7, 1, padding=3, bias=False)
+        self.ups.apply(init_weights)
+
+        if gin_channels != 0:
+            self.cond = nn.Conv1d(gin_channels, upsample_initial_channel, 1)
+
+    def forward(self, x, g=None):
+        x = self.conv_pre(x)
+        if g is not None:
+          x = x + self.cond(g)
+
+        for i in range(self.num_upsamples):
+            if self.decoder_alias_free and i >= self.decoder_alias_free_start_stage:
+                x = self.alias_free_pre_activations[i](x)
+            else:
+                x = F.leaky_relu(x, modules.LRELU_SLOPE)
+            x = self.ups[i](x)
+            xs = None
+            for j in range(self.num_kernels):
+                if xs is None:
+                    xs = self.resblocks[i*self.num_kernels+j](x)
+                else:
+                    xs += self.resblocks[i*self.num_kernels+j](x)
+            x = xs / self.num_kernels
+        if self.decoder_alias_free:
+            x = self.alias_free_post_activation(x)
+        else:
+            x = F.leaky_relu(x)
+        x = self.conv_post(x)
+        x = torch.tanh(x)
+
+        return x
+
+    def remove_weight_norm(self):
+        print('Removing weight norm...')
+        for l in self.ups:
+            remove_weight_norm(l)
+        for l in self.resblocks:
+            l.remove_weight_norm()
+
+
+class DiscriminatorP(torch.nn.Module):
+    def __init__(self, period, kernel_size=5, stride=3, use_spectral_norm=False):
+        super(DiscriminatorP, self).__init__()
+        self.period = period
+        self.use_spectral_norm = use_spectral_norm
+        norm_f = weight_norm if use_spectral_norm == False else spectral_norm
+        self.convs = nn.ModuleList([
+            norm_f(Conv2d(1, 32, (kernel_size, 1), (stride, 1), padding=(get_padding(kernel_size, 1), 0))),
+            norm_f(Conv2d(32, 128, (kernel_size, 1), (stride, 1), padding=(get_padding(kernel_size, 1), 0))),
+            norm_f(Conv2d(128, 512, (kernel_size, 1), (stride, 1), padding=(get_padding(kernel_size, 1), 0))),
+            norm_f(Conv2d(512, 1024, (kernel_size, 1), (stride, 1), padding=(get_padding(kernel_size, 1), 0))),
+            norm_f(Conv2d(1024, 1024, (kernel_size, 1), 1, padding=(get_padding(kernel_size, 1), 0))),
+        ])
+        self.conv_post = norm_f(Conv2d(1024, 1, (3, 1), 1, padding=(1, 0)))
+
+    def forward(self, x):
+        fmap = []
+
+        # 1d to 2d
+        b, c, t = x.shape
+        if t % self.period != 0: # pad first
+            n_pad = self.period - (t % self.period)
+            x = F.pad(x, (0, n_pad), "reflect")
+            t = t + n_pad
+        x = x.view(b, c, t // self.period, self.period)
+
+        for l in self.convs:
+            x = l(x)
+            x = F.leaky_relu(x, modules.LRELU_SLOPE)
+            fmap.append(x)
+        x = self.conv_post(x)
+        fmap.append(x)
+        x = torch.flatten(x, 1, -1)
+
+        return x, fmap
+
+
+class DiscriminatorS(torch.nn.Module):
+    def __init__(self, use_spectral_norm=False):
+        super(DiscriminatorS, self).__init__()
+        norm_f = weight_norm if use_spectral_norm == False else spectral_norm
+        self.convs = nn.ModuleList([
+            norm_f(Conv1d(1, 16, 15, 1, padding=7)),
+            norm_f(Conv1d(16, 64, 41, 4, groups=4, padding=20)),
+            norm_f(Conv1d(64, 256, 41, 4, groups=16, padding=20)),
+            norm_f(Conv1d(256, 1024, 41, 4, groups=64, padding=20)),
+            norm_f(Conv1d(1024, 1024, 41, 4, groups=256, padding=20)),
+            norm_f(Conv1d(1024, 1024, 5, 1, padding=2)),
+        ])
+        self.conv_post = norm_f(Conv1d(1024, 1, 3, 1, padding=1))
+
+    def forward(self, x):
+        fmap = []
+
+        for l in self.convs:
+            x = l(x)
+            x = F.leaky_relu(x, modules.LRELU_SLOPE)
+            fmap.append(x)
+        x = self.conv_post(x)
+        fmap.append(x)
+        x = torch.flatten(x, 1, -1)
+
+        return x, fmap
+
+
+class MultiPeriodDiscriminator(torch.nn.Module):
+    def __init__(self, use_spectral_norm=False):
+        super(MultiPeriodDiscriminator, self).__init__()
+        periods = [2,3,5,7,11]
+
+        discs = [DiscriminatorS(use_spectral_norm=use_spectral_norm)]
+        discs = discs + [DiscriminatorP(i, use_spectral_norm=use_spectral_norm) for i in periods]
+        self.discriminators = nn.ModuleList(discs)
+
+    def forward(self, y, y_hat):
+        y_d_rs = []
+        y_d_gs = []
+        fmap_rs = []
+        fmap_gs = []
+        for i, d in enumerate(self.discriminators):
+            y_d_r, fmap_r = d(y)
+            y_d_g, fmap_g = d(y_hat)
+            y_d_rs.append(y_d_r)
+            y_d_gs.append(y_d_g)
+            fmap_rs.append(fmap_r)
+            fmap_gs.append(fmap_g)
+
+        return y_d_rs, y_d_gs, fmap_rs, fmap_gs
+
+
+
+class SynthesizerTrn(nn.Module):
+  """
+  Synthesizer for Training
+  """
+
+  def __init__(self, 
+    n_vocab,
+    spec_channels,
+    segment_size,
+    inter_channels,
+    hidden_channels,
+    filter_channels,
+    n_heads,
+    n_layers,
+    kernel_size,
+    p_dropout,
+    resblock, 
+    resblock_kernel_sizes, 
+    resblock_dilation_sizes, 
+    upsample_rates, 
+    upsample_initial_channel, 
+    upsample_kernel_sizes,
+    n_speakers=0,
+    gin_channels=0,
+    use_sdp=True,
+    **kwargs):
+
+    super().__init__()
+    self.n_vocab = n_vocab
+    self.spec_channels = spec_channels
+    self.inter_channels = inter_channels
+    self.hidden_channels = hidden_channels
+    self.filter_channels = filter_channels
+    self.n_heads = n_heads
+    self.n_layers = n_layers
+    self.kernel_size = kernel_size
+    self.p_dropout = p_dropout
+    self.resblock = resblock
+    self.resblock_kernel_sizes = resblock_kernel_sizes
+    self.resblock_dilation_sizes = resblock_dilation_sizes
+    self.upsample_rates = upsample_rates
+    self.upsample_initial_channel = upsample_initial_channel
+    self.upsample_kernel_sizes = upsample_kernel_sizes
+    self.segment_size = segment_size
+    self.n_speakers = n_speakers
+    self.gin_channels = gin_channels
+
+    self.use_sdp = use_sdp
+
+    self.enc_p = TextEncoder(n_vocab,
+        inter_channels,
+        hidden_channels,
+        filter_channels,
+        n_heads,
+        n_layers,
+        kernel_size,
+        p_dropout)
+    self.dec = Generator(
+        inter_channels, resblock, resblock_kernel_sizes,
+        resblock_dilation_sizes, upsample_rates, upsample_initial_channel,
+        upsample_kernel_sizes, gin_channels=gin_channels,
+        decoder_alias_free=kwargs.get("decoder_alias_free", False),
+        decoder_alias_free_start_stage=kwargs.get("decoder_alias_free_start_stage", 2),
+        decoder_snake_logscale=kwargs.get("decoder_snake_logscale", True))
+    self.inference_only = bool(kwargs.get("inference_only", False))
+    if not self.inference_only:
+      self.enc_q = PosteriorEncoder(spec_channels, inter_channels, hidden_channels, 5, 1, 16, gin_channels=gin_channels)
+    self.flow = ResidualCouplingBlock(inter_channels, hidden_channels, 5, 1, 4, gin_channels=gin_channels)
+
+    if use_sdp:
+      self.dp = StochasticDurationPredictor(hidden_channels, 192, 3, 0.5, 4, gin_channels=gin_channels)
+    else:
+      self.dp = DurationPredictor(hidden_channels, 256, 3, 0.5, gin_channels=gin_channels)
+
+    if n_speakers > 1:
+      self.emb_g = nn.Embedding(n_speakers, gin_channels)
+
+  def forward(self, x, x_lengths, y, y_lengths, sid=None):
+
+    if self.inference_only:
+      raise RuntimeError("The public runtime is inference-only and has no posterior encoder.")
+
+    x, m_p, logs_p, x_mask = self.enc_p(x, x_lengths)
+    if self.n_speakers > 0:
+      g = self.emb_g(sid).unsqueeze(-1) # [b, h, 1]
+    else:
+      g = None
+
+    z, m_q, logs_q, y_mask = self.enc_q(y, y_lengths, g=g)
+    z_p = self.flow(z, y_mask, g=g)
+
+    with torch.no_grad():
+      # negative cross-entropy
+      s_p_sq_r = torch.exp(-2 * logs_p) # [b, d, t]
+      neg_cent1 = torch.sum(-0.5 * math.log(2 * math.pi) - logs_p, [1], keepdim=True) # [b, 1, t_s]
+      neg_cent2 = torch.matmul(-0.5 * (z_p ** 2).transpose(1, 2), s_p_sq_r) # [b, t_t, d] x [b, d, t_s] = [b, t_t, t_s]
+      neg_cent3 = torch.matmul(z_p.transpose(1, 2), (m_p * s_p_sq_r)) # [b, t_t, d] x [b, d, t_s] = [b, t_t, t_s]
+      neg_cent4 = torch.sum(-0.5 * (m_p ** 2) * s_p_sq_r, [1], keepdim=True) # [b, 1, t_s]
+      neg_cent = neg_cent1 + neg_cent2 + neg_cent3 + neg_cent4
+
+      attn_mask = torch.unsqueeze(x_mask, 2) * torch.unsqueeze(y_mask, -1)
+      attn = monotonic_align.maximum_path(neg_cent, attn_mask.squeeze(1)).unsqueeze(1).detach()
+
+    w = attn.sum(2)
+    if self.use_sdp:
+      l_length = self.dp(x, x_mask, w, g=g)
+      l_length = l_length / torch.sum(x_mask)
+    else:
+      logw_ = torch.log(w + 1e-6) * x_mask
+      logw = self.dp(x, x_mask, g=g)
+      l_length = torch.sum((logw - logw_)**2, [1,2]) / torch.sum(x_mask) # for averaging 
+
+    # expand prior
+    m_p = torch.matmul(attn.squeeze(1), m_p.transpose(1, 2)).transpose(1, 2)
+    logs_p = torch.matmul(attn.squeeze(1), logs_p.transpose(1, 2)).transpose(1, 2)
+
+    z_slice, ids_slice = commons.rand_slice_segments(z, y_lengths, self.segment_size)
+    o = self.dec(z_slice, g=g)
+    return o, l_length, attn, ids_slice, x_mask, y_mask, (z, z_p, m_p, logs_p, m_q, logs_q)
+
+  def infer(self, x, x_lengths, sid=None, noise_scale=1, length_scale=1, noise_scale_w=1., max_len=None):
+    x, m_p, logs_p, x_mask = self.enc_p(x, x_lengths)
+    if self.n_speakers > 0:
+      g = self.emb_g(sid).unsqueeze(-1) # [b, h, 1]
+    else:
+      g = None
+
+    if self.use_sdp:
+      logw = self.dp(x, x_mask, g=g, reverse=True, noise_scale=noise_scale_w)
+    else:
+      logw = self.dp(x, x_mask, g=g)
+    w = torch.exp(logw) * x_mask * length_scale
+    w_ceil = torch.ceil(w)
+    y_lengths = torch.clamp_min(torch.sum(w_ceil, [1, 2]), 1).long()
+    y_mask = torch.unsqueeze(commons.sequence_mask(y_lengths, None), 1).to(x_mask.dtype)
+    attn_mask = torch.unsqueeze(x_mask, 2) * torch.unsqueeze(y_mask, -1)
+    attn = commons.generate_path(w_ceil, attn_mask)
+
+    m_p = torch.matmul(attn.squeeze(1), m_p.transpose(1, 2)).transpose(1, 2) # [b, t', t], [b, t, d] -> [b, d, t']
+    logs_p = torch.matmul(attn.squeeze(1), logs_p.transpose(1, 2)).transpose(1, 2) # [b, t', t], [b, t, d] -> [b, d, t']
+
+    z_p = m_p + torch.randn_like(m_p) * torch.exp(logs_p) * noise_scale
+    z = self.flow(z_p, y_mask, g=g, reverse=True)
+    o = self.dec((z * y_mask)[:,:,:max_len], g=g)
+    return o, attn, y_mask, (z, z_p, m_p, logs_p)
+
+  def voice_conversion(self, y, y_lengths, sid_src, sid_tgt):
+    if self.inference_only:
+      raise RuntimeError("The public runtime is inference-only and does not support voice conversion.")
+    assert self.n_speakers > 0, "n_speakers have to be larger than 0."
+    g_src = self.emb_g(sid_src).unsqueeze(-1)
+    g_tgt = self.emb_g(sid_tgt).unsqueeze(-1)
+    z, m_q, logs_q, y_mask = self.enc_q(y, y_lengths, g=g_src)
+    z_p = self.flow(z, y_mask, g=g_src)
+    z_hat = self.flow(z_p, y_mask, g=g_tgt, reverse=True)
+    o_hat = self.dec(z_hat * y_mask, g=g_tgt)
+    return o_hat, y_mask, (z, z_p, z_hat)

runtime/modules.py

@@ -0,0 +1,390 @@
+import copy
+import math
+import numpy as np
+import scipy
+import torch
+from torch import nn
+from torch.nn import functional as F
+
+from torch.nn import Conv1d, ConvTranspose1d, AvgPool1d, Conv2d
+from torch.nn.utils import weight_norm, remove_weight_norm
+
+import commons
+from commons import init_weights, get_padding
+from transforms import piecewise_rational_quadratic_transform
+
+
+LRELU_SLOPE = 0.1
+
+
+class LayerNorm(nn.Module):
+  def __init__(self, channels, eps=1e-5):
+    super().__init__()
+    self.channels = channels
+    self.eps = eps
+
+    self.gamma = nn.Parameter(torch.ones(channels))
+    self.beta = nn.Parameter(torch.zeros(channels))
+
+  def forward(self, x):
+    x = x.transpose(1, -1)
+    x = F.layer_norm(x, (self.channels,), self.gamma, self.beta, self.eps)
+    return x.transpose(1, -1)
+
+ 
+class ConvReluNorm(nn.Module):
+  def __init__(self, in_channels, hidden_channels, out_channels, kernel_size, n_layers, p_dropout):
+    super().__init__()
+    self.in_channels = in_channels
+    self.hidden_channels = hidden_channels
+    self.out_channels = out_channels
+    self.kernel_size = kernel_size
+    self.n_layers = n_layers
+    self.p_dropout = p_dropout
+    assert n_layers > 1, "Number of layers should be larger than 0."
+
+    self.conv_layers = nn.ModuleList()
+    self.norm_layers = nn.ModuleList()
+    self.conv_layers.append(nn.Conv1d(in_channels, hidden_channels, kernel_size, padding=kernel_size//2))
+    self.norm_layers.append(LayerNorm(hidden_channels))
+    self.relu_drop = nn.Sequential(
+        nn.ReLU(),
+        nn.Dropout(p_dropout))
+    for _ in range(n_layers-1):
+      self.conv_layers.append(nn.Conv1d(hidden_channels, hidden_channels, kernel_size, padding=kernel_size//2))
+      self.norm_layers.append(LayerNorm(hidden_channels))
+    self.proj = nn.Conv1d(hidden_channels, out_channels, 1)
+    self.proj.weight.data.zero_()
+    self.proj.bias.data.zero_()
+
+  def forward(self, x, x_mask):
+    x_org = x
+    for i in range(self.n_layers):
+      x = self.conv_layers[i](x * x_mask)
+      x = self.norm_layers[i](x)
+      x = self.relu_drop(x)
+    x = x_org + self.proj(x)
+    return x * x_mask
+
+
+class DDSConv(nn.Module):
+  """
+  Dialted and Depth-Separable Convolution
+  """
+  def __init__(self, channels, kernel_size, n_layers, p_dropout=0.):
+    super().__init__()
+    self.channels = channels
+    self.kernel_size = kernel_size
+    self.n_layers = n_layers
+    self.p_dropout = p_dropout
+
+    self.drop = nn.Dropout(p_dropout)
+    self.convs_sep = nn.ModuleList()
+    self.convs_1x1 = nn.ModuleList()
+    self.norms_1 = nn.ModuleList()
+    self.norms_2 = nn.ModuleList()
+    for i in range(n_layers):
+      dilation = kernel_size ** i
+      padding = (kernel_size * dilation - dilation) // 2
+      self.convs_sep.append(nn.Conv1d(channels, channels, kernel_size, 
+          groups=channels, dilation=dilation, padding=padding
+      ))
+      self.convs_1x1.append(nn.Conv1d(channels, channels, 1))
+      self.norms_1.append(LayerNorm(channels))
+      self.norms_2.append(LayerNorm(channels))
+
+  def forward(self, x, x_mask, g=None):
+    if g is not None:
+      x = x + g
+    for i in range(self.n_layers):
+      y = self.convs_sep[i](x * x_mask)
+      y = self.norms_1[i](y)
+      y = F.gelu(y)
+      y = self.convs_1x1[i](y)
+      y = self.norms_2[i](y)
+      y = F.gelu(y)
+      y = self.drop(y)
+      x = x + y
+    return x * x_mask
+
+
+class WN(torch.nn.Module):
+  def __init__(self, hidden_channels, kernel_size, dilation_rate, n_layers, gin_channels=0, p_dropout=0):
+    super(WN, self).__init__()
+    assert(kernel_size % 2 == 1)
+    self.hidden_channels =hidden_channels
+    self.kernel_size = kernel_size,
+    self.dilation_rate = dilation_rate
+    self.n_layers = n_layers
+    self.gin_channels = gin_channels
+    self.p_dropout = p_dropout
+
+    self.in_layers = torch.nn.ModuleList()
+    self.res_skip_layers = torch.nn.ModuleList()
+    self.drop = nn.Dropout(p_dropout)
+
+    if gin_channels != 0:
+      cond_layer = torch.nn.Conv1d(gin_channels, 2*hidden_channels*n_layers, 1)
+      self.cond_layer = torch.nn.utils.weight_norm(cond_layer, name='weight')
+
+    for i in range(n_layers):
+      dilation = dilation_rate ** i
+      padding = int((kernel_size * dilation - dilation) / 2)
+      in_layer = torch.nn.Conv1d(hidden_channels, 2*hidden_channels, kernel_size,
+                                 dilation=dilation, padding=padding)
+      in_layer = torch.nn.utils.weight_norm(in_layer, name='weight')
+      self.in_layers.append(in_layer)
+
+      # last one is not necessary
+      if i < n_layers - 1:
+        res_skip_channels = 2 * hidden_channels
+      else:
+        res_skip_channels = hidden_channels
+
+      res_skip_layer = torch.nn.Conv1d(hidden_channels, res_skip_channels, 1)
+      res_skip_layer = torch.nn.utils.weight_norm(res_skip_layer, name='weight')
+      self.res_skip_layers.append(res_skip_layer)
+
+  def forward(self, x, x_mask, g=None, **kwargs):
+    output = torch.zeros_like(x)
+    n_channels_tensor = torch.IntTensor([self.hidden_channels])
+
+    if g is not None:
+      g = self.cond_layer(g)
+
+    for i in range(self.n_layers):
+      x_in = self.in_layers[i](x)
+      if g is not None:
+        cond_offset = i * 2 * self.hidden_channels
+        g_l = g[:,cond_offset:cond_offset+2*self.hidden_channels,:]
+      else:
+        g_l = torch.zeros_like(x_in)
+
+      acts = commons.fused_add_tanh_sigmoid_multiply(
+          x_in,
+          g_l,
+          n_channels_tensor)
+      acts = self.drop(acts)
+
+      res_skip_acts = self.res_skip_layers[i](acts)
+      if i < self.n_layers - 1:
+        res_acts = res_skip_acts[:,:self.hidden_channels,:]
+        x = (x + res_acts) * x_mask
+        output = output + res_skip_acts[:,self.hidden_channels:,:]
+      else:
+        output = output + res_skip_acts
+    return output * x_mask
+
+  def remove_weight_norm(self):
+    if self.gin_channels != 0:
+      torch.nn.utils.remove_weight_norm(self.cond_layer)
+    for l in self.in_layers:
+      torch.nn.utils.remove_weight_norm(l)
+    for l in self.res_skip_layers:
+     torch.nn.utils.remove_weight_norm(l)
+
+
+class ResBlock1(torch.nn.Module):
+    def __init__(self, channels, kernel_size=3, dilation=(1, 3, 5)):
+        super(ResBlock1, self).__init__()
+        self.convs1 = nn.ModuleList([
+            weight_norm(Conv1d(channels, channels, kernel_size, 1, dilation=dilation[0],
+                               padding=get_padding(kernel_size, dilation[0]))),
+            weight_norm(Conv1d(channels, channels, kernel_size, 1, dilation=dilation[1],
+                               padding=get_padding(kernel_size, dilation[1]))),
+            weight_norm(Conv1d(channels, channels, kernel_size, 1, dilation=dilation[2],
+                               padding=get_padding(kernel_size, dilation[2])))
+        ])
+        self.convs1.apply(init_weights)
+
+        self.convs2 = nn.ModuleList([
+            weight_norm(Conv1d(channels, channels, kernel_size, 1, dilation=1,
+                               padding=get_padding(kernel_size, 1))),
+            weight_norm(Conv1d(channels, channels, kernel_size, 1, dilation=1,
+                               padding=get_padding(kernel_size, 1))),
+            weight_norm(Conv1d(channels, channels, kernel_size, 1, dilation=1,
+                               padding=get_padding(kernel_size, 1)))
+        ])
+        self.convs2.apply(init_weights)
+
+    def forward(self, x, x_mask=None):
+        for c1, c2 in zip(self.convs1, self.convs2):
+            xt = F.leaky_relu(x, LRELU_SLOPE)
+            if x_mask is not None:
+                xt = xt * x_mask
+            xt = c1(xt)
+            xt = F.leaky_relu(xt, LRELU_SLOPE)
+            if x_mask is not None:
+                xt = xt * x_mask
+            xt = c2(xt)
+            x = xt + x
+        if x_mask is not None:
+            x = x * x_mask
+        return x
+
+    def remove_weight_norm(self):
+        for l in self.convs1:
+            remove_weight_norm(l)
+        for l in self.convs2:
+            remove_weight_norm(l)
+
+
+class ResBlock2(torch.nn.Module):
+    def __init__(self, channels, kernel_size=3, dilation=(1, 3)):
+        super(ResBlock2, self).__init__()
+        self.convs = nn.ModuleList([
+            weight_norm(Conv1d(channels, channels, kernel_size, 1, dilation=dilation[0],
+                               padding=get_padding(kernel_size, dilation[0]))),
+            weight_norm(Conv1d(channels, channels, kernel_size, 1, dilation=dilation[1],
+                               padding=get_padding(kernel_size, dilation[1])))
+        ])
+        self.convs.apply(init_weights)
+
+    def forward(self, x, x_mask=None):
+        for c in self.convs:
+            xt = F.leaky_relu(x, LRELU_SLOPE)
+            if x_mask is not None:
+                xt = xt * x_mask
+            xt = c(xt)
+            x = xt + x
+        if x_mask is not None:
+            x = x * x_mask
+        return x
+
+    def remove_weight_norm(self):
+        for l in self.convs:
+            remove_weight_norm(l)
+
+
+class Log(nn.Module):
+  def forward(self, x, x_mask, reverse=False, **kwargs):
+    if not reverse:
+      y = torch.log(torch.clamp_min(x, 1e-5)) * x_mask
+      logdet = torch.sum(-y, [1, 2])
+      return y, logdet
+    else:
+      x = torch.exp(x) * x_mask
+      return x
+    
+
+class Flip(nn.Module):
+  def forward(self, x, *args, reverse=False, **kwargs):
+    x = torch.flip(x, [1])
+    if not reverse:
+      logdet = torch.zeros(x.size(0)).to(dtype=x.dtype, device=x.device)
+      return x, logdet
+    else:
+      return x
+
+
+class ElementwiseAffine(nn.Module):
+  def __init__(self, channels):
+    super().__init__()
+    self.channels = channels
+    self.m = nn.Parameter(torch.zeros(channels,1))
+    self.logs = nn.Parameter(torch.zeros(channels,1))
+
+  def forward(self, x, x_mask, reverse=False, **kwargs):
+    if not reverse:
+      y = self.m + torch.exp(self.logs) * x
+      y = y * x_mask
+      logdet = torch.sum(self.logs * x_mask, [1,2])
+      return y, logdet
+    else:
+      x = (x - self.m) * torch.exp(-self.logs) * x_mask
+      return x
+
+
+class ResidualCouplingLayer(nn.Module):
+  def __init__(self,
+      channels,
+      hidden_channels,
+      kernel_size,
+      dilation_rate,
+      n_layers,
+      p_dropout=0,
+      gin_channels=0,
+      mean_only=False):
+    assert channels % 2 == 0, "channels should be divisible by 2"
+    super().__init__()
+    self.channels = channels
+    self.hidden_channels = hidden_channels
+    self.kernel_size = kernel_size
+    self.dilation_rate = dilation_rate
+    self.n_layers = n_layers
+    self.half_channels = channels // 2
+    self.mean_only = mean_only
+
+    self.pre = nn.Conv1d(self.half_channels, hidden_channels, 1)
+    self.enc = WN(hidden_channels, kernel_size, dilation_rate, n_layers, p_dropout=p_dropout, gin_channels=gin_channels)
+    self.post = nn.Conv1d(hidden_channels, self.half_channels * (2 - mean_only), 1)
+    self.post.weight.data.zero_()
+    self.post.bias.data.zero_()
+
+  def forward(self, x, x_mask, g=None, reverse=False):
+    x0, x1 = torch.split(x, [self.half_channels]*2, 1)
+    h = self.pre(x0) * x_mask
+    h = self.enc(h, x_mask, g=g)
+    stats = self.post(h) * x_mask
+    if not self.mean_only:
+      m, logs = torch.split(stats, [self.half_channels]*2, 1)
+    else:
+      m = stats
+      logs = torch.zeros_like(m)
+
+    if not reverse:
+      x1 = m + x1 * torch.exp(logs) * x_mask
+      x = torch.cat([x0, x1], 1)
+      logdet = torch.sum(logs, [1,2])
+      return x, logdet
+    else:
+      x1 = (x1 - m) * torch.exp(-logs) * x_mask
+      x = torch.cat([x0, x1], 1)
+      return x
+
+
+class ConvFlow(nn.Module):
+  def __init__(self, in_channels, filter_channels, kernel_size, n_layers, num_bins=10, tail_bound=5.0):
+    super().__init__()
+    self.in_channels = in_channels
+    self.filter_channels = filter_channels
+    self.kernel_size = kernel_size
+    self.n_layers = n_layers
+    self.num_bins = num_bins
+    self.tail_bound = tail_bound
+    self.half_channels = in_channels // 2
+
+    self.pre = nn.Conv1d(self.half_channels, filter_channels, 1)
+    self.convs = DDSConv(filter_channels, kernel_size, n_layers, p_dropout=0.)
+    self.proj = nn.Conv1d(filter_channels, self.half_channels * (num_bins * 3 - 1), 1)
+    self.proj.weight.data.zero_()
+    self.proj.bias.data.zero_()
+
+  def forward(self, x, x_mask, g=None, reverse=False):
+    x0, x1 = torch.split(x, [self.half_channels]*2, 1)
+    h = self.pre(x0)
+    h = self.convs(h, x_mask, g=g)
+    h = self.proj(h) * x_mask
+
+    b, c, t = x0.shape
+    h = h.reshape(b, c, -1, t).permute(0, 1, 3, 2) # [b, cx?, t] -> [b, c, t, ?]
+
+    unnormalized_widths = h[..., :self.num_bins] / math.sqrt(self.filter_channels)
+    unnormalized_heights = h[..., self.num_bins:2*self.num_bins] / math.sqrt(self.filter_channels)
+    unnormalized_derivatives = h[..., 2 * self.num_bins:]
+
+    x1, logabsdet = piecewise_rational_quadratic_transform(x1,
+        unnormalized_widths,
+        unnormalized_heights,
+        unnormalized_derivatives,
+        inverse=reverse,
+        tails='linear',
+        tail_bound=self.tail_bound
+    )
+
+    x = torch.cat([x0, x1], 1) * x_mask
+    logdet = torch.sum(logabsdet * x_mask, [1,2])
+    if not reverse:
+        return x, logdet
+    else:
+        return x

runtime/monotonic_align.py

@@ -0,0 +1,4 @@
+"""Training-only alignment stub; deployable inference never calls maximum_path."""
+
+def maximum_path(*args, **kwargs):
+    raise RuntimeError("Monotonic alignment is unavailable in the inference package.")

runtime/text/LICENSE

@@ -0,0 +1,19 @@
+Copyright (c) 2017 Keith Ito
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

runtime/text/__init__.py

@@ -0,0 +1,54 @@
+""" from https://github.com/keithito/tacotron """
+from text import cleaners
+from text.symbols import symbols
+
+
+# Mappings from symbol to numeric ID and vice versa:
+_symbol_to_id = {s: i for i, s in enumerate(symbols)}
+_id_to_symbol = {i: s for i, s in enumerate(symbols)}
+
+
+def text_to_sequence(text, cleaner_names):
+  '''Converts a string of text to a sequence of IDs corresponding to the symbols in the text.
+    Args:
+      text: string to convert to a sequence
+      cleaner_names: names of the cleaner functions to run the text through
+    Returns:
+      List of integers corresponding to the symbols in the text
+  '''
+  sequence = []
+
+  clean_text = _clean_text(text, cleaner_names)
+  for symbol in clean_text:
+    symbol_id = _symbol_to_id[symbol]
+    sequence += [symbol_id]
+  return sequence
+
+
+def cleaned_text_to_sequence(cleaned_text):
+  '''Converts a string of text to a sequence of IDs corresponding to the symbols in the text.
+    Args:
+      text: string to convert to a sequence
+    Returns:
+      List of integers corresponding to the symbols in the text
+  '''
+  sequence = [_symbol_to_id[symbol] for symbol in cleaned_text]
+  return sequence
+
+
+def sequence_to_text(sequence):
+  '''Converts a sequence of IDs back to a string'''
+  result = ''
+  for symbol_id in sequence:
+    s = _id_to_symbol[symbol_id]
+    result += s
+  return result
+
+
+def _clean_text(text, cleaner_names):
+  for name in cleaner_names:
+    cleaner = getattr(cleaners, name)
+    if not cleaner:
+      raise Exception('Unknown cleaner: %s' % name)
+    text = cleaner(text)
+  return text

runtime/text/cleaners.py

@@ -0,0 +1,100 @@
+""" from https://github.com/keithito/tacotron """
+
+'''
+Cleaners are transformations that run over the input text at both training and eval time.
+
+Cleaners can be selected by passing a comma-delimited list of cleaner names as the "cleaners"
+hyperparameter. Some cleaners are English-specific. You'll typically want to use:
+  1. "english_cleaners" for English text
+  2. "transliteration_cleaners" for non-English text that can be transliterated to ASCII using
+     the Unidecode library (https://pypi.python.org/pypi/Unidecode)
+  3. "basic_cleaners" if you do not want to transliterate (in this case, you should also update
+     the symbols in symbols.py to match your data).
+'''
+
+import re
+from unidecode import unidecode
+from phonemizer import phonemize
+
+
+# Regular expression matching whitespace:
+_whitespace_re = re.compile(r'\s+')
+
+# List of (regular expression, replacement) pairs for abbreviations:
+_abbreviations = [(re.compile('\\b%s\\.' % x[0], re.IGNORECASE), x[1]) for x in [
+  ('mrs', 'misess'),
+  ('mr', 'mister'),
+  ('dr', 'doctor'),
+  ('st', 'saint'),
+  ('co', 'company'),
+  ('jr', 'junior'),
+  ('maj', 'major'),
+  ('gen', 'general'),
+  ('drs', 'doctors'),
+  ('rev', 'reverend'),
+  ('lt', 'lieutenant'),
+  ('hon', 'honorable'),
+  ('sgt', 'sergeant'),
+  ('capt', 'captain'),
+  ('esq', 'esquire'),
+  ('ltd', 'limited'),
+  ('col', 'colonel'),
+  ('ft', 'fort'),
+]]
+
+
+def expand_abbreviations(text):
+  for regex, replacement in _abbreviations:
+    text = re.sub(regex, replacement, text)
+  return text
+
+
+def expand_numbers(text):
+  return normalize_numbers(text)
+
+
+def lowercase(text):
+  return text.lower()
+
+
+def collapse_whitespace(text):
+  return re.sub(_whitespace_re, ' ', text)
+
+
+def convert_to_ascii(text):
+  return unidecode(text)
+
+
+def basic_cleaners(text):
+  '''Basic pipeline that lowercases and collapses whitespace without transliteration.'''
+  text = lowercase(text)
+  text = collapse_whitespace(text)
+  return text
+
+
+def transliteration_cleaners(text):
+  '''Pipeline for non-English text that transliterates to ASCII.'''
+  text = convert_to_ascii(text)
+  text = lowercase(text)
+  text = collapse_whitespace(text)
+  return text
+
+
+def english_cleaners(text):
+  '''Pipeline for English text, including abbreviation expansion.'''
+  text = convert_to_ascii(text)
+  text = lowercase(text)
+  text = expand_abbreviations(text)
+  phonemes = phonemize(text, language='en-us', backend='espeak', strip=True)
+  phonemes = collapse_whitespace(phonemes)
+  return phonemes
+
+
+def english_cleaners2(text):
+  '''Pipeline for English text, including abbreviation expansion. + punctuation + stress'''
+  text = convert_to_ascii(text)
+  text = lowercase(text)
+  text = expand_abbreviations(text)
+  phonemes = phonemize(text, language='en-us', backend='espeak', strip=True, preserve_punctuation=True, with_stress=True)
+  phonemes = collapse_whitespace(phonemes)
+  return phonemes

runtime/text/symbols.py

@@ -0,0 +1,16 @@
+""" from https://github.com/keithito/tacotron """
+
+'''
+Defines the set of symbols used in text input to the model.
+'''
+_pad        = '_'
+_punctuation = ';:,.!?¡¿—…"«»“” '
+_letters = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz'
+_letters_ipa = "ɑɐɒæɓʙβɔɕçɗɖðʤəɘɚɛɜɝɞɟʄɡɠɢʛɦɧħɥʜɨɪʝɭɬɫɮʟɱɯɰŋɳɲɴøɵɸθœɶʘɹɺɾɻʀʁɽʂʃʈʧʉʊʋⱱʌɣɤʍχʎʏʑʐʒʔʡʕʢǀǁǂǃˈˌːˑʼʴʰʱʲʷˠˤ˞↓↑→↗↘'̩'ᵻ"
+
+
+# Export all symbols:
+symbols = [_pad] + list(_punctuation) + list(_letters) + list(_letters_ipa)
+
+# Special symbol ids
+SPACE_ID = symbols.index(" ")

runtime/transforms.py

@@ -0,0 +1,193 @@
+import torch
+from torch.nn import functional as F
+
+import numpy as np
+
+
+DEFAULT_MIN_BIN_WIDTH = 1e-3
+DEFAULT_MIN_BIN_HEIGHT = 1e-3
+DEFAULT_MIN_DERIVATIVE = 1e-3
+
+
+def piecewise_rational_quadratic_transform(inputs, 
+                                           unnormalized_widths,
+                                           unnormalized_heights,
+                                           unnormalized_derivatives,
+                                           inverse=False,
+                                           tails=None, 
+                                           tail_bound=1.,
+                                           min_bin_width=DEFAULT_MIN_BIN_WIDTH,
+                                           min_bin_height=DEFAULT_MIN_BIN_HEIGHT,
+                                           min_derivative=DEFAULT_MIN_DERIVATIVE):
+
+    if tails is None:
+        spline_fn = rational_quadratic_spline
+        spline_kwargs = {}
+    else:
+        spline_fn = unconstrained_rational_quadratic_spline
+        spline_kwargs = {
+            'tails': tails,
+            'tail_bound': tail_bound
+        }
+
+    outputs, logabsdet = spline_fn(
+            inputs=inputs,
+            unnormalized_widths=unnormalized_widths,
+            unnormalized_heights=unnormalized_heights,
+            unnormalized_derivatives=unnormalized_derivatives,
+            inverse=inverse,
+            min_bin_width=min_bin_width,
+            min_bin_height=min_bin_height,
+            min_derivative=min_derivative,
+            **spline_kwargs
+    )
+    return outputs, logabsdet
+
+
+def searchsorted(bin_locations, inputs, eps=1e-6):
+    bin_locations[..., -1] += eps
+    return torch.sum(
+        inputs[..., None] >= bin_locations,
+        dim=-1
+    ) - 1
+
+
+def unconstrained_rational_quadratic_spline(inputs,
+                                            unnormalized_widths,
+                                            unnormalized_heights,
+                                            unnormalized_derivatives,
+                                            inverse=False,
+                                            tails='linear',
+                                            tail_bound=1.,
+                                            min_bin_width=DEFAULT_MIN_BIN_WIDTH,
+                                            min_bin_height=DEFAULT_MIN_BIN_HEIGHT,
+                                            min_derivative=DEFAULT_MIN_DERIVATIVE):
+    inside_interval_mask = (inputs >= -tail_bound) & (inputs <= tail_bound)
+    outside_interval_mask = ~inside_interval_mask
+
+    outputs = torch.zeros_like(inputs)
+    logabsdet = torch.zeros_like(inputs)
+
+    if tails == 'linear':
+        unnormalized_derivatives = F.pad(unnormalized_derivatives, pad=(1, 1))
+        constant = np.log(np.exp(1 - min_derivative) - 1)
+        unnormalized_derivatives[..., 0] = constant
+        unnormalized_derivatives[..., -1] = constant
+
+        outputs[outside_interval_mask] = inputs[outside_interval_mask]
+        logabsdet[outside_interval_mask] = 0
+    else:
+        raise RuntimeError('{} tails are not implemented.'.format(tails))
+
+    outputs[inside_interval_mask], logabsdet[inside_interval_mask] = rational_quadratic_spline(
+        inputs=inputs[inside_interval_mask],
+        unnormalized_widths=unnormalized_widths[inside_interval_mask, :],
+        unnormalized_heights=unnormalized_heights[inside_interval_mask, :],
+        unnormalized_derivatives=unnormalized_derivatives[inside_interval_mask, :],
+        inverse=inverse,
+        left=-tail_bound, right=tail_bound, bottom=-tail_bound, top=tail_bound,
+        min_bin_width=min_bin_width,
+        min_bin_height=min_bin_height,
+        min_derivative=min_derivative
+    )
+
+    return outputs, logabsdet
+
+def rational_quadratic_spline(inputs,
+                              unnormalized_widths,
+                              unnormalized_heights,
+                              unnormalized_derivatives,
+                              inverse=False,
+                              left=0., right=1., bottom=0., top=1.,
+                              min_bin_width=DEFAULT_MIN_BIN_WIDTH,
+                              min_bin_height=DEFAULT_MIN_BIN_HEIGHT,
+                              min_derivative=DEFAULT_MIN_DERIVATIVE):
+    if torch.min(inputs) < left or torch.max(inputs) > right:
+        raise ValueError('Input to a transform is not within its domain')
+
+    num_bins = unnormalized_widths.shape[-1]
+
+    if min_bin_width * num_bins > 1.0:
+        raise ValueError('Minimal bin width too large for the number of bins')
+    if min_bin_height * num_bins > 1.0:
+        raise ValueError('Minimal bin height too large for the number of bins')
+
+    widths = F.softmax(unnormalized_widths, dim=-1)
+    widths = min_bin_width + (1 - min_bin_width * num_bins) * widths
+    cumwidths = torch.cumsum(widths, dim=-1)
+    cumwidths = F.pad(cumwidths, pad=(1, 0), mode='constant', value=0.0)
+    cumwidths = (right - left) * cumwidths + left
+    cumwidths[..., 0] = left
+    cumwidths[..., -1] = right
+    widths = cumwidths[..., 1:] - cumwidths[..., :-1]
+
+    derivatives = min_derivative + F.softplus(unnormalized_derivatives)
+
+    heights = F.softmax(unnormalized_heights, dim=-1)
+    heights = min_bin_height + (1 - min_bin_height * num_bins) * heights
+    cumheights = torch.cumsum(heights, dim=-1)
+    cumheights = F.pad(cumheights, pad=(1, 0), mode='constant', value=0.0)
+    cumheights = (top - bottom) * cumheights + bottom
+    cumheights[..., 0] = bottom
+    cumheights[..., -1] = top
+    heights = cumheights[..., 1:] - cumheights[..., :-1]
+
+    if inverse:
+        bin_idx = searchsorted(cumheights, inputs)[..., None]
+    else:
+        bin_idx = searchsorted(cumwidths, inputs)[..., None]
+
+    input_cumwidths = cumwidths.gather(-1, bin_idx)[..., 0]
+    input_bin_widths = widths.gather(-1, bin_idx)[..., 0]
+
+    input_cumheights = cumheights.gather(-1, bin_idx)[..., 0]
+    delta = heights / widths
+    input_delta = delta.gather(-1, bin_idx)[..., 0]
+
+    input_derivatives = derivatives.gather(-1, bin_idx)[..., 0]
+    input_derivatives_plus_one = derivatives[..., 1:].gather(-1, bin_idx)[..., 0]
+
+    input_heights = heights.gather(-1, bin_idx)[..., 0]
+
+    if inverse:
+        a = (((inputs - input_cumheights) * (input_derivatives
+                                             + input_derivatives_plus_one
+                                             - 2 * input_delta)
+              + input_heights * (input_delta - input_derivatives)))
+        b = (input_heights * input_derivatives
+             - (inputs - input_cumheights) * (input_derivatives
+                                              + input_derivatives_plus_one
+                                              - 2 * input_delta))
+        c = - input_delta * (inputs - input_cumheights)
+
+        discriminant = b.pow(2) - 4 * a * c
+        assert (discriminant >= 0).all()
+
+        root = (2 * c) / (-b - torch.sqrt(discriminant))
+        outputs = root * input_bin_widths + input_cumwidths
+
+        theta_one_minus_theta = root * (1 - root)
+        denominator = input_delta + ((input_derivatives + input_derivatives_plus_one - 2 * input_delta)
+                                     * theta_one_minus_theta)
+        derivative_numerator = input_delta.pow(2) * (input_derivatives_plus_one * root.pow(2)
+                                                     + 2 * input_delta * theta_one_minus_theta
+                                                     + input_derivatives * (1 - root).pow(2))
+        logabsdet = torch.log(derivative_numerator) - 2 * torch.log(denominator)
+
+        return outputs, -logabsdet
+    else:
+        theta = (inputs - input_cumwidths) / input_bin_widths
+        theta_one_minus_theta = theta * (1 - theta)
+
+        numerator = input_heights * (input_delta * theta.pow(2)
+                                     + input_derivatives * theta_one_minus_theta)
+        denominator = input_delta + ((input_derivatives + input_derivatives_plus_one - 2 * input_delta)
+                                     * theta_one_minus_theta)
+        outputs = input_cumheights + numerator / denominator
+
+        derivative_numerator = input_delta.pow(2) * (input_derivatives_plus_one * theta.pow(2)
+                                                     + 2 * input_delta * theta_one_minus_theta
+                                                     + input_derivatives * (1 - theta).pow(2))
+        logabsdet = torch.log(derivative_numerator) - 2 * torch.log(denominator)
+
+        return outputs, logabsdet

runtime/utils.py

@@ -0,0 +1,256 @@
+import os
+import glob
+import sys
+import argparse
+import logging
+import json
+import subprocess
+import numpy as np
+from scipy.io.wavfile import read
+import torch
+
+MATPLOTLIB_FLAG = False
+
+logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)
+logger = logging
+
+
+def load_checkpoint(checkpoint_path, model, optimizer=None):
+  assert os.path.isfile(checkpoint_path)
+  checkpoint_dict = torch.load(checkpoint_path, map_location='cpu')
+  iteration = checkpoint_dict['iteration']
+  learning_rate = checkpoint_dict['learning_rate']
+  if optimizer is not None:
+    optimizer.load_state_dict(checkpoint_dict['optimizer'])
+  saved_state_dict = checkpoint_dict['model']
+  if hasattr(model, 'module'):
+    state_dict = model.module.state_dict()
+  else:
+    state_dict = model.state_dict()
+  new_state_dict= {}
+  for k, v in state_dict.items():
+    try:
+      new_state_dict[k] = saved_state_dict[k]
+    except:
+      logger.info("%s is not in the checkpoint" % k)
+      new_state_dict[k] = v
+  if hasattr(model, 'module'):
+    model.module.load_state_dict(new_state_dict)
+  else:
+    model.load_state_dict(new_state_dict)
+  logger.info("Loaded checkpoint '{}' (iteration {})" .format(
+    checkpoint_path, iteration))
+  return model, optimizer, learning_rate, iteration
+
+
+def save_checkpoint(model, optimizer, learning_rate, iteration, checkpoint_path):
+  logger.info("Saving model and optimizer state at iteration {} to {}".format(
+    iteration, checkpoint_path))
+  if hasattr(model, 'module'):
+    state_dict = model.module.state_dict()
+  else:
+    state_dict = model.state_dict()
+  torch.save({'model': state_dict,
+              'iteration': iteration,
+              'optimizer': optimizer.state_dict(),
+              'learning_rate': learning_rate}, checkpoint_path)
+
+
+def summarize(writer, global_step, scalars={}, histograms={}, images={}, audios={}, audio_sampling_rate=22050):
+  for k, v in scalars.items():
+    writer.add_scalar(k, v, global_step)
+  for k, v in histograms.items():
+    writer.add_histogram(k, v, global_step)
+  for k, v in images.items():
+    writer.add_image(k, v, global_step, dataformats='HWC')
+  for k, v in audios.items():
+    writer.add_audio(k, v, global_step, audio_sampling_rate)
+
+
+def latest_checkpoint_path(dir_path, regex="G_*.pth"):
+  f_list = glob.glob(os.path.join(dir_path, regex))
+  f_list.sort(key=lambda f: int("".join(filter(str.isdigit, f))))
+  x = f_list[-1]
+  print(x)
+  return x
+
+
+def plot_spectrogram_to_numpy(spectrogram):
+  global MATPLOTLIB_FLAG
+  if not MATPLOTLIB_FLAG:
+    import matplotlib
+    matplotlib.use("Agg")
+    MATPLOTLIB_FLAG = True
+    mpl_logger = logging.getLogger('matplotlib')
+    mpl_logger.setLevel(logging.WARNING)
+  import matplotlib.pylab as plt
+  import numpy as np
+  
+  fig, ax = plt.subplots(figsize=(10,2))
+  im = ax.imshow(spectrogram, aspect="auto", origin="lower",
+                  interpolation='none')
+  plt.colorbar(im, ax=ax)
+  plt.xlabel("Frames")
+  plt.ylabel("Channels")
+  plt.tight_layout()
+
+  fig.canvas.draw()
+  data = np.asarray(fig.canvas.buffer_rgba(), dtype=np.uint8)[..., :3].copy()
+  plt.close()
+  return data
+
+
+def plot_alignment_to_numpy(alignment, info=None):
+  global MATPLOTLIB_FLAG
+  if not MATPLOTLIB_FLAG:
+    import matplotlib
+    matplotlib.use("Agg")
+    MATPLOTLIB_FLAG = True
+    mpl_logger = logging.getLogger('matplotlib')
+    mpl_logger.setLevel(logging.WARNING)
+  import matplotlib.pylab as plt
+  import numpy as np
+
+  fig, ax = plt.subplots(figsize=(6, 4))
+  im = ax.imshow(alignment.transpose(), aspect='auto', origin='lower',
+                  interpolation='none')
+  fig.colorbar(im, ax=ax)
+  xlabel = 'Decoder timestep'
+  if info is not None:
+      xlabel += '\n\n' + info
+  plt.xlabel(xlabel)
+  plt.ylabel('Encoder timestep')
+  plt.tight_layout()
+
+  fig.canvas.draw()
+  data = np.asarray(fig.canvas.buffer_rgba(), dtype=np.uint8)[..., :3].copy()
+  plt.close()
+  return data
+
+
+def load_wav_to_torch(full_path):
+  sampling_rate, data = read(full_path)
+  return torch.FloatTensor(data.astype(np.float32)), sampling_rate
+
+
+def load_filepaths_and_text(filename, split="|"):
+  with open(filename, encoding='utf-8') as f:
+    filepaths_and_text = [line.strip().split(split) for line in f]
+  return filepaths_and_text
+
+
+def get_hparams(init=True):
+  parser = argparse.ArgumentParser()
+  parser.add_argument('-c', '--config', type=str, default="./configs/base.json",
+                      help='JSON file for configuration')
+  parser.add_argument('-m', '--model', type=str, required=True,
+                      help='Model name')
+  
+  args = parser.parse_args()
+  model_dir = os.path.join("./logs", args.model)
+
+  if not os.path.exists(model_dir):
+    os.makedirs(model_dir)
+
+  config_path = args.config
+  config_save_path = os.path.join(model_dir, "config.json")
+  if init:
+    with open(config_path, "r") as f:
+      data = f.read()
+    with open(config_save_path, "w") as f:
+      f.write(data)
+  else:
+    with open(config_save_path, "r") as f:
+      data = f.read()
+  config = json.loads(data)
+  
+  hparams = HParams(**config)
+  hparams.model_dir = model_dir
+  return hparams
+
+
+def get_hparams_from_dir(model_dir):
+  config_save_path = os.path.join(model_dir, "config.json")
+  with open(config_save_path, "r") as f:
+    data = f.read()
+  config = json.loads(data)
+
+  hparams =HParams(**config)
+  hparams.model_dir = model_dir
+  return hparams
+
+
+def get_hparams_from_file(config_path):
+  with open(config_path, "r") as f:
+    data = f.read()
+  config = json.loads(data)
+
+  hparams =HParams(**config)
+  return hparams
+
+
+def check_git_hash(model_dir):
+  source_dir = os.path.dirname(os.path.realpath(__file__))
+  if not os.path.exists(os.path.join(source_dir, ".git")):
+    logger.warn("{} is not a git repository, therefore hash value comparison will be ignored.".format(
+      source_dir
+    ))
+    return
+
+  cur_hash = subprocess.getoutput("git rev-parse HEAD")
+
+  path = os.path.join(model_dir, "githash")
+  if os.path.exists(path):
+    saved_hash = open(path).read()
+    if saved_hash != cur_hash:
+      logger.warn("git hash values are different. {}(saved) != {}(current)".format(
+        saved_hash[:8], cur_hash[:8]))
+  else:
+    open(path, "w").write(cur_hash)
+
+
+def get_logger(model_dir, filename="train.log"):
+  global logger
+  logger = logging.getLogger(os.path.basename(model_dir))
+  logger.setLevel(logging.DEBUG)
+  
+  formatter = logging.Formatter("%(asctime)s\t%(name)s\t%(levelname)s\t%(message)s")
+  if not os.path.exists(model_dir):
+    os.makedirs(model_dir)
+  h = logging.FileHandler(os.path.join(model_dir, filename))
+  h.setLevel(logging.DEBUG)
+  h.setFormatter(formatter)
+  logger.addHandler(h)
+  return logger
+
+
+class HParams():
+  def __init__(self, **kwargs):
+    for k, v in kwargs.items():
+      if type(v) == dict:
+        v = HParams(**v)
+      self[k] = v
+    
+  def keys(self):
+    return self.__dict__.keys()
+
+  def items(self):
+    return self.__dict__.items()
+
+  def values(self):
+    return self.__dict__.values()
+
+  def __len__(self):
+    return len(self.__dict__)
+
+  def __getitem__(self, key):
+    return getattr(self, key)
+
+  def __setitem__(self, key, value):
+    return setattr(self, key, value)
+
+  def __contains__(self, key):
+    return key in self.__dict__
+
+  def __repr__(self):
+    return self.__dict__.__repr__()

samples/clara_intro.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ea6341d6b631f3c2898503e30c5c6a0a6ee4f7e5fcaf4baac22f271d77f139ed
+size 494636

samples/pip_intro.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc6c66ac08a3bc6a533aae6fcde059ec30ad6386af92801f350be374122f0118
+size 327212

samples/scrappy_intro.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dd0ebed0b0a2bbeedbb4849309e7b67c49e269eb8b00ba9527dd5d2daf08ac98
+size 525868

samples/silas_intro.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3c315e12d7235e2a957a90166b5d1ab44b9c49c68955ca7fb1072399540b5a02
+size 420396

speakers.json

@@ -0,0 +1,27 @@
+{
+  "n_speakers": 4,
+  "speakers": {
+    "scrappy": 0,
+    "clara": 1,
+    "silas": 2,
+    "pip": 3
+  },
+  "counts": {
+    "scrappy": {
+      "train": 4122,
+      "val": 216
+    },
+    "clara": {
+      "train": 4155,
+      "val": 218
+    },
+    "silas": {
+      "train": 4150,
+      "val": 218
+    },
+    "pip": {
+      "train": 4119,
+      "val": 216
+    }
+  }
+}

third_party/ALIAS_FREE_TORCH_LICENSE.txt

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

third_party/BIGVGAN_LICENSE.txt

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 NVIDIA CORPORATION.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

third_party/VITS_LICENSE.txt

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2021 Jaehyeon Kim
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

trainer/eval_candidate_ms.py

@@ -0,0 +1,87 @@
+#!/usr/bin/env python3
+"""Synthesize eval prompts from a MULTI-SPEAKER training candidate.
+
+Same flow as eval_candidate.py, two differences: the model is rebuilt from a
+config that carries n_speakers/gin_channels (train_ms.py writes
+<run-dir>/config_ms.json — the package config.json alone reconstructs the
+single-speaker graph and the strict load fails), and infer() gets a sid.
+"""
+import argparse
+import json
+import os
+import sys
+from pathlib import Path
+
+import soundfile as sf
+import torch
+
+BASE = Path(__file__).resolve().parents[1]
+PKG = Path(os.environ.get("INFLECT_PKG", BASE / "Inflect-Micro-v2"))
+sys.path.insert(0, str(PKG / "runtime"))
+sys.path.insert(0, str(PKG))
+
+import commons  # noqa: E402
+import utils  # noqa: E402
+from inference import edge_fade, optimize_for_inference  # noqa: E402
+from inflect_vits_frontend import run_vits_frontend  # noqa: E402
+from models import SynthesizerTrn  # noqa: E402
+from text import cleaned_text_to_sequence  # noqa: E402
+from text.symbols import symbols  # noqa: E402
+
+PROMPTS = [
+    ("held1", "The committee will publish its findings on the fourteenth of October."),
+    ("held2", "Beneath the ice, the ocean kept its own kind of time."),
+    ("held3", "First, check the seal. Second, note the pressure. Finally, log both numbers."),
+    ("held4", "Was it courage, or simply the absence of any other option?"),
+    ("held5", "The fleet in this house runs the same quality inference that once required a data center."),
+]
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--candidate", required=True, help="ms candidate .pth from train_ms.py")
+    ap.add_argument("--sid", required=True, type=int, help="speaker id (see speakers.json)")
+    ap.add_argument("--config", default=None,
+                    help="config with model.n_speakers/gin_channels; default: config_ms.json two dirs up from the candidate (the run dir)")
+    ap.add_argument("--out", required=True)
+    ap.add_argument("--seed", type=int, default=7)
+    args = ap.parse_args()
+
+    cfg = Path(args.config) if args.config else Path(args.candidate).resolve().parent.parent / "config_ms.json"
+    assert cfg.exists(), f"no config at {cfg} — pass --config <run-dir>/config_ms.json"
+    hps = utils.get_hparams_from_file(str(cfg))
+    model_cfg = dict(hps.model)
+    n_speakers = model_cfg.get("n_speakers", 0)
+    assert n_speakers >= 2, f"{cfg} has n_speakers={n_speakers} — not a multi-speaker config"
+    assert 0 <= args.sid < n_speakers, f"--sid {args.sid} out of range for n_speakers={n_speakers}"
+
+    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+    net = SynthesizerTrn(len(symbols), hps.data.filter_length // 2 + 1,
+                         hps.train.segment_size // hps.data.hop_length,
+                         **model_cfg).to(device).eval()
+    ck = torch.load(args.candidate, map_location="cpu", weights_only=True)
+    net.load_state_dict(ck["model"], strict=True)
+    optimize_for_inference(net)
+    sid = torch.LongTensor([args.sid]).to(device)
+
+    out = Path(args.out)
+    out.mkdir(parents=True, exist_ok=True)
+    (out / "prompts.tsv").write_text("\n".join(f"{k}\t{t}" for k, t in PROMPTS) + "\n")
+    with torch.inference_mode():
+        for key, text in PROMPTS:
+            phonemes = run_vits_frontend(text).phoneme_text
+            seq = cleaned_text_to_sequence(phonemes)
+            if hps.data.add_blank:
+                seq = commons.intersperse(seq, 0)
+            tokens = torch.LongTensor(seq).unsqueeze(0).to(device)
+            lengths = torch.LongTensor([tokens.size(1)]).to(device)
+            torch.manual_seed(args.seed)
+            wav = net.infer(tokens, lengths, sid=sid, noise_scale=0.667, noise_scale_w=0.8,
+                            length_scale=1.0)[0][0, 0].float().cpu().numpy()
+            sf.write(out / f"{key}.wav", edge_fade(wav, hps.data.sampling_rate), hps.data.sampling_rate)
+            print(f"{key} (sid {args.sid}): {len(wav)/hps.data.sampling_rate:.1f}s")
+    print(f"eval wavs → {out}")
+
+
+if __name__ == "__main__":
+    main()

trainer/prep_filelists_ms.py

@@ -0,0 +1,121 @@
+#!/usr/bin/env python3
+"""Multi-corpus survivors → multi-speaker filelists (wav_path|speaker_id|phoneme_text).
+
+Each corpus contributes one speaker: DIR/survivors.tsv (the QC pass-list) is
+phonemized with the package's exact frontend — same as prep_filelists.py — and
+OOV-symbol lines are dropped. The val split is stratified PER SPEAKER so a
+small second corpus keeps val coverage. Writes filelist_train_ms.txt,
+filelist_val_ms.txt and speakers.json (the name→id map train/eval bind to).
+
+Speaker ids must be exactly 0..N-1 with N >= 2: upstream models.py creates
+emb_g only for n_speakers > 1 but uses it for n_speakers > 0, so 1 crashes.
+
+Corpora may reuse clip ids (scrappy2 renders the same nar_* ids as corpus/) —
+the absolute wav path is the only unique key, never merge on bare id.
+"""
+import argparse
+import json
+import os
+import random
+import sys
+from pathlib import Path
+
+BASE = Path(__file__).resolve().parents[1]
+PKG = Path(os.environ.get("INFLECT_PKG", BASE / "Inflect-Micro-v2"))
+sys.path.insert(0, str(PKG / "runtime"))
+sys.path.insert(0, str(PKG))
+
+from inflect_vits_frontend import run_vits_frontend_batch  # noqa: E402
+from text.symbols import symbols  # noqa: E402
+
+KNOWN = set(symbols)
+
+
+def parse_specs(args):
+    """[(dir, sid, name, survivors, wavs), ...] from --corpus and/or --spec."""
+    specs = []
+    for c in args.corpus or []:
+        parts = c.split(":")
+        if len(parts) != 3:
+            sys.exit(f"--corpus wants DIR:SID:NAME, got {c!r}")
+        d = Path(parts[0]).resolve()
+        specs.append({"dir": d, "sid": int(parts[1]), "name": parts[2],
+                      "survivors": d / "survivors.tsv", "wavs": d / "wavs24k"})
+    if args.spec:
+        for e in json.loads(Path(args.spec).read_text()):
+            d = Path(e["dir"]).resolve()
+            specs.append({"dir": d, "sid": int(e["sid"]), "name": e["name"],
+                          "survivors": (d / e["survivors"]).resolve() if "survivors" in e else d / "survivors.tsv",
+                          "wavs": (d / e["wavs"]).resolve() if "wavs" in e else d / "wavs24k"})
+    if len(specs) < 2:
+        sys.exit(f"need >= 2 speakers (n_speakers=1 crashes upstream), got {len(specs)}")
+    sids = sorted(s["sid"] for s in specs)
+    if sids != list(range(len(specs))):
+        sys.exit(f"speaker ids must be exactly 0..{len(specs) - 1}, got {sids}")
+    if len({s["name"] for s in specs}) != len(specs):
+        sys.exit("duplicate speaker names")
+    for s in specs:
+        if not s["survivors"].exists():
+            sys.exit(f"{s['name']}: no QC pass-list at {s['survivors']} — run scripts/qc.py first")
+        if not s["wavs"].is_dir():
+            sys.exit(f"{s['name']}: no wav dir at {s['wavs']}")
+    return specs
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--corpus", action="append", metavar="DIR:SID:NAME",
+                    help="repeatable, e.g. --corpus corpus:0:narrator --corpus corpus-clara:1:clara")
+    ap.add_argument("--spec", help="JSON list of {dir, sid, name[, survivors, wavs]} (alternative to --corpus)")
+    ap.add_argument("--val-frac", type=float, default=0.05)
+    ap.add_argument("--seed", type=int, default=7)
+    ap.add_argument("--out-dir", default=str(BASE / "corpus-ms"))
+    args = ap.parse_args()
+
+    specs = parse_specs(args)
+    rng = random.Random(args.seed)
+    train, val, counts = [], [], {}
+    for s in specs:
+        rows = [l.rstrip("\n").split("\t", 1) for l in open(s["survivors"]) if "\t" in l]
+        texts = [t for _, t in rows]
+        print(f"[{s['name']} sid={s['sid']}] phonemizing {len(rows)} lines via Inflect frontend…")
+        outs = run_vits_frontend_batch(texts, jobs=4)
+
+        entries, oov = [], 0
+        for (fid, _), out in zip(rows, outs):
+            wav = s["wavs"] / f"{fid}.wav"
+            if not (wav.exists() and out.phoneme_text):
+                continue
+            bad = set(out.phoneme_text) - KNOWN
+            if bad:
+                oov += 1
+                continue
+            entries.append(f"{wav}|{s['sid']}|{out.phoneme_text}")
+        if oov:
+            print(f"[{s['name']}] dropped {oov} lines with out-of-vocabulary phoneme symbols")
+
+        rng.shuffle(entries)
+        n_val = max(4, int(len(entries) * args.val_frac))
+        if len(entries) - n_val < 1:
+            sys.exit(f"[{s['name']}] only {len(entries)} usable clips — not enough to train")
+        val += entries[:n_val]
+        train += entries[n_val:]
+        counts[s["name"]] = {"train": len(entries) - n_val, "val": n_val}
+        print(f"[{s['name']}] {len(entries) - n_val} train / {n_val} val")
+
+    rng.shuffle(train)
+    rng.shuffle(val)
+    out_dir = Path(args.out_dir)
+    out_dir.mkdir(parents=True, exist_ok=True)
+    (out_dir / "filelist_train_ms.txt").write_text("\n".join(train) + "\n")
+    (out_dir / "filelist_val_ms.txt").write_text("\n".join(val) + "\n")
+    meta = {"n_speakers": len(specs),
+            "speakers": {s["name"]: s["sid"] for s in specs},
+            "corpora": {s["name"]: str(s["dir"]) for s in specs},
+            "counts": counts}
+    (out_dir / "speakers.json").write_text(json.dumps(meta, indent=2) + "\n")
+    print(f"{len(train)} train / {len(val)} val ({len(specs)} speakers) → {out_dir}/filelist_*_ms.txt + speakers.json")
+
+
+if __name__ == "__main__":
+    main()

trainer/train_ms.py

@@ -0,0 +1,379 @@
+#!/usr/bin/env python3
+"""Warm-start multi-speaker fine-tune of Inflect-Micro-v2 (N voices, one model).
+
+Same recipe as train_ft.py but VITS multi-speaker: SynthesizerTrn is built with
+n_speakers/gin_channels so the runtime's retained hooks light up — emb_g plus
+the g-conditioning already wired into enc_q (WN cond_layer), flow (per-coupling
+cond_layer), dp (cond) and dec (cond). enc_p takes no g, same as upstream VITS.
+
+Warm-start from the single-speaker model.pth with strict=False: every base
+tensor keeps its shape (gin conditioning is additive), so 100% of the base
+loads and ONLY {enc_q.*, emb_g.*, dec.cond.*, dp.cond.*, flow cond_layer.*}
+initialize fresh — anything else missing fails loudly (see warm_start_ms).
+
+Filelists are 3-column wav_path|speaker_id|phoneme_text from
+prep_filelists_ms.py. Spec caches are redirected OUT of the corpora dirs
+(train_ft.py writes .spec.pt next to the wavs; here corpora stay read-only —
+a live render may be writing next to them).
+"""
+import argparse
+import hashlib
+import json
+import math
+import os
+import sys
+import time
+from collections import Counter
+from pathlib import Path
+
+import numpy as np
+import soundfile as sf
+import torch
+import torch.nn.functional as F
+from torch.utils.data import DataLoader, Dataset
+
+BASE = Path(__file__).resolve().parents[1]
+PKG = Path(os.environ.get("INFLECT_PKG", BASE / "Inflect-Micro-v2"))
+sys.path.insert(0, str(PKG / "runtime"))
+sys.path.insert(0, str(PKG))
+
+import commons  # noqa: E402
+import utils  # noqa: E402
+from models import MultiPeriodDiscriminator, SynthesizerTrn  # noqa: E402
+from text import cleaned_text_to_sequence  # noqa: E402
+from text.symbols import symbols  # noqa: E402
+
+# ------------------------------------------- transforms (same as train_ft.py)
+_mel_basis = {}
+_hann = {}
+
+
+def _window(win_size, device, dtype):
+    key = f"{win_size}_{device}_{dtype}"
+    if key not in _hann:
+        _hann[key] = torch.hann_window(win_size).to(device=device, dtype=dtype)
+    return _hann[key]
+
+
+def spectrogram_torch(y, n_fft, hop_size, win_size):
+    y = F.pad(y.unsqueeze(1), (int((n_fft - hop_size) / 2), int((n_fft - hop_size) / 2)),
+              mode="reflect").squeeze(1)
+    spec = torch.stft(y, n_fft, hop_length=hop_size, win_length=win_size,
+                      window=_window(win_size, y.device, y.dtype),
+                      center=False, pad_mode="reflect", normalized=False,
+                      onesided=True, return_complex=True)
+    return torch.sqrt(spec.real ** 2 + spec.imag ** 2 + 1e-6)
+
+
+def _mel_fb(n_fft, num_mels, sr, fmin, fmax, device, dtype):
+    key = f"{n_fft}_{num_mels}_{fmax}_{device}_{dtype}"
+    if key not in _mel_basis:
+        from torchaudio.functional import melscale_fbanks
+        # slaney/slaney == librosa_mel_fn defaults used by upstream VITS
+        fb = melscale_fbanks(n_fft // 2 + 1, fmin, fmax, num_mels, sr,
+                             norm="slaney", mel_scale="slaney").T
+        _mel_basis[key] = fb.to(device=device, dtype=dtype)
+    return _mel_basis[key]
+
+
+def spec_to_mel_torch(spec, n_fft, num_mels, sr, fmin, fmax):
+    mel = torch.matmul(_mel_fb(n_fft, num_mels, sr, fmin, fmax, spec.device, spec.dtype), spec)
+    return torch.log(torch.clamp(mel, min=1e-5))
+
+
+def mel_spectrogram_torch(y, n_fft, num_mels, sr, hop_size, win_size, fmin, fmax):
+    return spec_to_mel_torch(spectrogram_torch(y, n_fft, hop_size, win_size),
+                             n_fft, num_mels, sr, fmin, fmax)
+
+
+# ----------------------------------------------- losses (same as train_ft.py)
+def feature_loss(fmap_r, fmap_g):
+    loss = 0
+    for dr, dg in zip(fmap_r, fmap_g):
+        for rl, gl in zip(dr, dg):
+            loss += torch.mean(torch.abs(rl.float().detach() - gl.float()))
+    return loss * 2
+
+
+def discriminator_loss(disc_real, disc_gen):
+    loss = 0
+    for dr, dg in zip(disc_real, disc_gen):
+        loss += torch.mean((1 - dr.float()) ** 2) + torch.mean(dg.float() ** 2)
+    return loss
+
+
+def generator_loss(disc_outputs):
+    loss = 0
+    for dg in disc_outputs:
+        loss += torch.mean((1 - dg.float()) ** 2)
+    return loss
+
+
+def kl_loss(z_p, logs_q, m_p, logs_p, z_mask):
+    z_p, logs_q = z_p.float(), logs_q.float()
+    m_p, logs_p = m_p.float(), logs_p.float()
+    z_mask = z_mask.float()
+    kl = logs_p - logs_q - 0.5
+    kl += 0.5 * ((z_p - m_p) ** 2) * torch.exp(-2.0 * logs_p)
+    return torch.sum(kl * z_mask) / torch.sum(z_mask)
+
+
+# ---------------------------------------------------------------- warm-start
+MS_FRESH_PREFIXES = ("enc_q.", "emb_g.", "dec.cond.", "dp.cond.")
+
+
+def is_expected_fresh(key):
+    """Keys that legitimately have no source tensor in the single-speaker base:
+    the whole posterior encoder (base ck is inference-only — already fresh in
+    train_ft.py), the speaker embedding, and every gin cond layer (dec.cond,
+    dp.cond, flow.flows.{0,2,4,6}.enc.cond_layer.{bias,weight_g,weight_v})."""
+    if key.startswith(MS_FRESH_PREFIXES):
+        return True
+    return key.startswith("flow.") and ".cond_layer." in key
+
+
+def warm_start_ms(net_g, base_sd):
+    """strict=False load with an EXPLICIT whitelist — logs every fresh key and
+    fails loudly on anything missing outside it (or unexpected in the ck)."""
+    missing, unexpected = net_g.load_state_dict(base_sd, strict=False)
+    fresh_ok = sorted(k for k in missing if is_expected_fresh(k))
+    fresh_bad = sorted(k for k in missing if not is_expected_fresh(k))
+    n_total = len(net_g.state_dict())
+    print(f"warm-start: {len(base_sd)} tensors offered, {n_total - len(missing)} loaded, "
+          f"{len(missing)} fresh, {len(unexpected)} unexpected")
+    for k in fresh_ok:
+        print(f"  fresh (expected): {k}")
+    for k in fresh_bad:
+        print(f"  MISSING (not in multi-speaker whitelist): {k}")
+    for k in unexpected:
+        print(f"  UNEXPECTED in checkpoint: {k}")
+    assert not unexpected, f"unexpected keys in base checkpoint: {unexpected}"
+    assert not fresh_bad, f"missing keys outside the multi-speaker whitelist: {fresh_bad}"
+    assert any(k.startswith("emb_g.") for k in missing), \
+        "emb_g not fresh — n_speakers/gin_channels did not take effect"
+    return missing, unexpected
+
+
+# ---------------------------------------------------------------- data
+class TextAudioSpeakerDataset(Dataset):
+    """3-column filelist: wav_path|speaker_id|phoneme_text (split on first TWO
+    pipes). Spec cache goes to spec_cache_dir keyed by wav-path hash — never
+    next to the wavs (corpora stay read-only)."""
+
+    def __init__(self, filelist, hps, spec_cache_dir):
+        self.items = [l.strip().split("|", 2) for l in open(filelist) if l.count("|") >= 2]
+        self.hps = hps
+        self.cache = Path(spec_cache_dir)
+        self.cache.mkdir(parents=True, exist_ok=True)
+
+    def __len__(self):
+        return len(self.items)
+
+    def __getitem__(self, idx):
+        wav_path, sid, phonemes = self.items[idx]
+        seq = cleaned_text_to_sequence(phonemes)
+        if self.hps.data.add_blank:
+            seq = commons.intersperse(seq, 0)
+        text = torch.LongTensor(seq)
+        audio, sr = sf.read(wav_path, dtype="float32")
+        assert sr == self.hps.data.sampling_rate, f"{wav_path}: {sr}"
+        audio = torch.from_numpy(audio).float().unsqueeze(0)
+        st = os.stat(wav_path)
+        spec_path = self.cache / f"{hashlib.sha1(f'{wav_path}:{st.st_size}:{st.st_mtime_ns}'.encode()).hexdigest()}.spec.pt"
+        try:
+            spec = torch.load(spec_path, weights_only=True)
+        except (FileNotFoundError, RuntimeError):
+            spec = spectrogram_torch(audio, self.hps.data.filter_length,
+                                     self.hps.data.hop_length, self.hps.data.win_length).squeeze(0)
+            torch.save(spec, spec_path)
+        return text, spec, audio.squeeze(0), int(sid)
+
+
+def collate(batch):
+    batch = sorted(batch, key=lambda x: x[1].size(1), reverse=True)
+    max_t = max(x[0].size(0) for x in batch)
+    max_s = max(x[1].size(1) for x in batch)
+    max_w = max(x[2].size(0) for x in batch)
+    n = len(batch)
+    text = torch.zeros(n, max_t, dtype=torch.long)
+    spec = torch.zeros(n, batch[0][1].size(0), max_s)
+    wav = torch.zeros(n, 1, max_w)
+    tl, sl, wl = (torch.zeros(n, dtype=torch.long) for _ in range(3))
+    sid = torch.LongTensor([x[3] for x in batch])
+    for i, (t, s, w, _) in enumerate(batch):
+        text[i, :t.size(0)] = t
+        spec[i, :, :s.size(1)] = s
+        wav[i, 0, :w.size(0)] = w
+        tl[i], sl[i], wl[i] = t.size(0), s.size(1), w.size(0)
+    return text, tl, spec, sl, wav, wl, sid
+
+
+# ---------------------------------------------------------------- export
+def export_candidate(net_g, step, lr, out_dir, orig_ck_format):
+    sd = {k: v for k, v in net_g.state_dict().items() if not k.startswith("enc_q.")}
+    out = {"format": orig_ck_format, "model": sd, "iteration": step,
+           "learning_rate": lr, "deployable_parameters": sum(v.numel() for v in sd.values())}
+    path = Path(out_dir) / f"candidate_{step:06d}.pth"
+    torch.save(out, path)
+    return path
+
+
+# ---------------------------------------------------------------- main
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--train-filelist", default=str(BASE / "corpus-ms/filelist_train_ms.txt"))
+    ap.add_argument("--val-filelist", default=str(BASE / "corpus-ms/filelist_val_ms.txt"))
+    ap.add_argument("--speakers", default=str(BASE / "corpus-ms/speakers.json"))
+    ap.add_argument("--n-speakers", type=int, default=None,
+                    help="default: n_speakers from --speakers json (must be >= 2)")
+    ap.add_argument("--gin-channels", type=int, default=256)
+    ap.add_argument("--spec-cache-dir", default=None, help="default: <run-dir>/spec_cache")
+    ap.add_argument("--run-dir", default=str(BASE / "runs/ms-pilot"))
+    ap.add_argument("--batch-size", type=int, default=24)
+    ap.add_argument("--lr-g", type=float, default=1e-4)
+    ap.add_argument("--lr-d", type=float, default=2e-4)
+    ap.add_argument("--lr-gamma", type=float, default=0.9999)
+    ap.add_argument("--freeze-dec-steps", type=int, default=1500)
+    ap.add_argument("--max-steps", type=int, default=12000)
+    ap.add_argument("--export-every", type=int, default=1000)
+    ap.add_argument("--log-every", type=int, default=50)
+    ap.add_argument("--c-mel", type=float, default=45.0)
+    ap.add_argument("--c-kl", type=float, default=1.0)
+    args = ap.parse_args()
+
+    if args.n_speakers is None:
+        meta = json.loads(Path(args.speakers).read_text())
+        args.n_speakers = int(meta["n_speakers"])
+        print(f"speakers: {meta['speakers']}")
+    assert args.n_speakers >= 2, \
+        f"n_speakers must be >= 2 (upstream creates emb_g only for > 1 but uses it for > 0), got {args.n_speakers}"
+
+    device = torch.device("cuda")
+    run_dir = Path(args.run_dir)
+    (run_dir / "candidates").mkdir(parents=True, exist_ok=True)
+    hps = utils.get_hparams_from_file(str(PKG / "config.json"))
+    orig_ck = torch.load(PKG / "model.pth", map_location="cpu", weights_only=True)
+    ck_format = orig_ck.get("format", "inflect_v2_inference_config_v1")
+
+    # deployable config for MS candidates: same as the package's, plus the two
+    # keys inference needs to rebuild this graph (eval_candidate_ms.py reads it)
+    deploy_cfg = json.loads((PKG / "config.json").read_text())
+    deploy_cfg["model"]["n_speakers"] = args.n_speakers
+    deploy_cfg["model"]["gin_channels"] = args.gin_channels
+    deploy_cfg["data"]["n_speakers"] = args.n_speakers
+    (run_dir / "config_ms.json").write_text(json.dumps(deploy_cfg, indent=2) + "\n")
+
+    model_cfg = dict(hps.model)
+    model_cfg["inference_only"] = False
+    model_cfg["n_speakers"] = args.n_speakers
+    model_cfg["gin_channels"] = args.gin_channels
+    seg_frames = hps.train.segment_size // hps.data.hop_length
+    net_g = SynthesizerTrn(len(symbols), hps.data.filter_length // 2 + 1,
+                           seg_frames, **model_cfg).to(device)
+    net_d = MultiPeriodDiscriminator(model_cfg.get("use_spectral_norm", False)).to(device)
+
+    warm_start_ms(net_g, orig_ck["model"])
+    # dec.cond/dp.cond are plain Conv1d: zero-init makes them identity at step 0
+    # so the frozen decoder hears the converged base unperturbed. The WN
+    # cond_layers stay default init — zeroing weight_v under weight_norm is unsafe.
+    torch.nn.init.zeros_(net_g.dec.cond.weight)
+    torch.nn.init.zeros_(net_g.dec.cond.bias)
+    torch.nn.init.zeros_(net_g.dp.cond.weight)
+    torch.nn.init.zeros_(net_g.dp.cond.bias)
+
+    opt_g = torch.optim.AdamW(net_g.parameters(), args.lr_g, betas=(0.8, 0.99), eps=1e-9)
+    opt_d = torch.optim.AdamW(net_d.parameters(), args.lr_d, betas=(0.8, 0.99), eps=1e-9)
+    sched_g = torch.optim.lr_scheduler.ExponentialLR(opt_g, gamma=args.lr_gamma)
+    sched_d = torch.optim.lr_scheduler.ExponentialLR(opt_d, gamma=args.lr_gamma)
+
+    spec_cache = args.spec_cache_dir or (run_dir / "spec_cache")
+    ds = TextAudioSpeakerDataset(args.train_filelist, hps, spec_cache)
+    counts = Counter(int(s) for _, s, _ in ds.items)
+    assert sorted(counts) == list(range(args.n_speakers)), \
+        f"train filelist must cover every sid 0..{args.n_speakers - 1} exactly, got {sorted(counts)}"
+    dl = DataLoader(ds, batch_size=args.batch_size, shuffle=True, drop_last=True,
+                    collate_fn=collate, num_workers=4, persistent_workers=True)
+    print(f"dataset: {len(ds)} clips, {len(dl)} steps/epoch, batch {args.batch_size}, "
+          f"per-speaker: {dict(sorted(counts.items()))}")
+
+    net_g.dec.requires_grad_(False)
+    net_g.dec.cond.requires_grad_(True)  # fresh speaker-cond layer trains from step 0
+    dec_frozen = True
+    print(f"decoder FROZEN for first {args.freeze_dec_steps} steps (dec.cond stays live)")
+
+    step, t0 = 0, time.perf_counter()
+    log = (run_dir / "train_log.jsonl").open("a")
+    net_g.train()
+    net_d.train()
+    while step < args.max_steps:
+        for text, tl, spec, sl, wav, wl, sid in dl:
+            if step >= args.max_steps:
+                break
+            if dec_frozen and step >= args.freeze_dec_steps:
+                net_g.dec.requires_grad_(True)
+                dec_frozen = False
+                print(f"step {step}: decoder UNFROZEN")
+            text, tl = text.to(device), tl.to(device)
+            spec, sl = spec.to(device), sl.to(device)
+            wav, sid = wav.to(device), sid.to(device)
+
+            y_hat, l_length, attn, ids_slice, x_mask, z_mask, \
+                (z, z_p, m_p, logs_p, m_q, logs_q) = net_g(text, tl, spec, sl, sid=sid)
+
+            mel = spec_to_mel_torch(spec, hps.data.filter_length, hps.data.n_mel_channels,
+                                    hps.data.sampling_rate, hps.data.mel_fmin, hps.data.mel_fmax)
+            y_mel = commons.slice_segments(mel, ids_slice, seg_frames)
+            y_hat_mel = mel_spectrogram_torch(y_hat.squeeze(1), hps.data.filter_length,
+                                              hps.data.n_mel_channels, hps.data.sampling_rate,
+                                              hps.data.hop_length, hps.data.win_length,
+                                              hps.data.mel_fmin, hps.data.mel_fmax)
+            y = commons.slice_segments(wav, ids_slice * hps.data.hop_length, hps.train.segment_size)
+
+            # discriminator
+            y_d_hat_r, y_d_hat_g, _, _ = net_d(y, y_hat.detach())
+            loss_disc = discriminator_loss(y_d_hat_r, y_d_hat_g)
+            opt_d.zero_grad()
+            loss_disc.backward()
+            commons.clip_grad_value_(net_d.parameters(), None)
+            opt_d.step()
+
+            # generator
+            y_d_hat_r, y_d_hat_g, fmap_r, fmap_g = net_d(y, y_hat)
+            loss_dur = torch.sum(l_length.float())
+            loss_mel = F.l1_loss(y_mel, y_hat_mel) * args.c_mel
+            loss_kl = kl_loss(z_p, logs_q, m_p, logs_p, z_mask) * args.c_kl
+            loss_fm = feature_loss(fmap_r, fmap_g)
+            loss_gen = generator_loss(y_d_hat_g)
+            loss_all = loss_gen + loss_fm + loss_mel + loss_dur + loss_kl
+            opt_g.zero_grad()
+            loss_all.backward()
+            commons.clip_grad_value_(net_g.parameters(), None)
+            opt_g.step()
+            sched_g.step()
+            sched_d.step()
+            step += 1
+
+            if step % args.log_every == 0:
+                rate = step / (time.perf_counter() - t0)
+                rec = {"step": step, "g": round(loss_all.item(), 3),
+                       "mel": round(loss_mel.item(), 3), "dur": round(loss_dur.item(), 4),
+                       "kl": round(loss_kl.item(), 3), "d": round(loss_disc.item(), 3),
+                       "lr": sched_g.get_last_lr()[0], "steps_per_s": round(rate, 2)}
+                print(json.dumps(rec), flush=True)
+                log.write(json.dumps(rec) + "\n")
+                log.flush()
+
+            if step % args.export_every == 0 or step == args.max_steps:
+                utils.save_checkpoint(net_g, opt_g, sched_g.get_last_lr()[0], step,
+                                      str(run_dir / "G_latest.pth"))
+                utils.save_checkpoint(net_d, opt_d, sched_d.get_last_lr()[0], step,
+                                      str(run_dir / "D_latest.pth"))
+                p = export_candidate(net_g, step, sched_g.get_last_lr()[0],
+                                     run_dir / "candidates", ck_format)
+                print(f"exported {p}", flush=True)
+
+    print("training complete")
+
+
+if __name__ == "__main__":
+    main()