scrappylabsai/scrappy-voice-family-1

Scrappy Voice Family — four voices in one 10M-param CPU model

Four distinct narrators — Scrappy, Clara, Silas, and Pip — sharing a single 10.26M-parameter / 41MB checkpoint that runs on a plain CPU, no GPU, no cloud. It's a multi-speaker fine-tune of owensong/Inflect-Micro-v2 (Apache-2.0), warm-started from our first voice and trained on all four corpora pooled together.

Listen: samples/scrappy_intro.wav · samples/clara_intro.wav · samples/silas_intro.wav · samples/pip_intro.wav — each voice introducing itself, all four generated on a desktop CPU by this one checkpoint.

Voice	id	Character
Scrappy	0	Warm, expressive narrator — the original ScrappyLabs voice
Clara	1	Clear professional female narrator, broadcast polish, neutral American with a warm undertone
Silas	2	Deep, authoritative older male — gravelly, slow, movie-trailer gravitas
Pip	3	Bright, energetic young American female — quick, playful, podcast-host diction

🎮 Try all four voices — and blend between them: interactive demo on Spaces — the Space was built and gifted to us by the Hugging Face team. Thanks, HF 🤗

Why a family instead of four models

We shipped scrappy-voice-1 as a single-voice distill, then built three more voices the same way. The obvious next question: does one shared model hold four identities without smearing them together?

It does — and pooling the data made it better, not worse.

Same recipe as the solo voices, end to end:

1. A teacher renders each corpus. Clara, Silas and Pip did not exist before this — they were described, not recorded. A voice-design model running on our own hardware turned a single sentence per voice ("a deep, authoritative older male narrator, gravelly texture, slow and weighty, movie-trailer gravitas" is all of Silas's origin) into a speaking voice. Each teacher then read 4,400 short clips (5.5h @ 24kHz) from a text corpus we control — so every transcript is known by construction. Scrappy, the eldest, was distilled earlier from a commercial cloud narrator voice.
2. An ASR gate cleans it. Every clip is round-tripped through speech recognition and scored against its transcript (≥0.85 word overlap required), alongside signal checks for clipping, silence and duration. Pass rates: Clara 99.4%, Silas 99.3%, Pip 98.5% (Scrappy's original corpus: 98.6%). This is the check that catches audio which says the wrong thing beautifully.
3. Warm-start, multi-speaker. n_speakers=4, gin_channels=256. Rather than start from the stock base, we warm-started from Scrappy's own 50k checkpoint — chosen by a measured A/B, not vibes: 26.4 vs 28.2 final mel loss at 12k steps, with ASR intelligibility at parity. An already-distilled voice is a better launch pad than the stock release.
4. 75,000 steps, batch 24, LR 5e-5, fp32, over 16,546 pooled training clips (plus 868 held out) across the four voices.

Final mel loss 18.4–18.9 — comfortably below the Clara (19.4) and Pip (21.0) solo runs and level with Silas's 18.1, the best of the three. Four voices sharing one model landed at roughly the quality of the best single-voice run rather than paying for the company, which is the pleasant surprise of the whole exercise. Multi-speaker conditioning also costs essentially nothing at train time: 4.36 steps/s, same as single-speaker.

Held-out intelligibility on the family model (ASR word-overlap against the intro script): Clara 1.00, Pip 1.00, Scrappy 0.96, Silas 0.92 — mean 0.97. The solo models score 1.00 for both Silas and Clara, so a little per-voice sharpness is the price of sharing; see Honest limitations.

Usage

Drop-in for the packaged runtime, with two additions: voice= and blend=.

from inference import InflectTTS

tts = InflectTTS(model_dir=".", device="cpu")

print(tts.voices)   # ['scrappy', 'clara', 'silas', 'pip']  (ordered by speaker id)

# pick a voice by name (case-insensitive) or by id
tts.save("Clear, professional, and easy to listen to.", "clara.wav", voice="clara", seed=7)
tts.save("Low, slow, and built for weight.",            "silas.wav", voice=2)

# blend two or more voices — weights are normalized for you
tts.save("Somewhere between the two of them.", "morph.wav",
         blend={"clara": 0.7, "silas": 0.3}, seed=7)

sample_rate, audio = tts.synthesize("Returns numpy, if you'd rather.", voice="pip")

python inference.py --model-dir . --device cpu --voice clara \
  --text "Hello from the family." --output out.wav

python inference.py --blend "clara=0.7, silas=0.3" \
  --text "And this is a blend." --output morph.wav

voice accepts a name, an int speaker id, or a digit string; omit it and you get voice 0 (Scrappy). blend takes a {voice: weight} dict, normalizes the weights to sum to 1, and overrides voice when both are given. Everything else is unchanged from upstream: English only, deterministic seeds, punctuation-aware long-form chunking, speed 0.5–2.0, variation 0.0–1.0. Write numbers out as words for best results.

Why four voices don't cost four models

Each voice is one 256-float speaker-embedding row — about 1KB. The multi-speaker machinery (the conditioning pathways through the flow, decoder and posterior encoder) is a one-time ~0.9M-parameter cost over the single-voice model; after that, voice number five would add roughly a kilobyte, not another 37MB. That's the whole argument for a family: you pay for the plumbing once.

Blending works because those embedding rows live in a continuous space. A weighted average of two rows is a valid conditioning vector, so {"clara": 0.7, "silas": 0.3} renders a voice that genuinely sits between them rather than crossfading two renders. Weights are normalized, duplicates that resolve to the same voice are summed, and negative, non-finite, all-zero or unknown-name inputs are rejected outright.

Honest limitations

• Blends near the middle can get uncanny. Only the four anchor voices were trained. The space between them is interpolation, not supervision — light blends (say 80/20) usually sound like a plausible person; 50/50 blends of very different voices (Silas and Pip, for instance) can land somewhere no human throat goes. Audition before you ship one.
• Sharing costs a little per-voice sharpness. Silas scores 0.92 held-out intelligibility in the family model vs 1.00 solo. If you need one voice at maximum fidelity and don't care about the others, the solo models still win narrowly.
• Prosody is where distillation loses the most. Timbre and identity transfer well; the teacher's long-range timing instincts — dramatic pauses, phrase-level planning — get averaged. The duration predictor is the smallest organ in a VITS, so expect a flatter read than the source voices.
• espeak-ng mispronounces "Silas" as /siːləz/. Spell it Sighlus in synthesis input to get /saɪləs/. A frontend quirk, not a model one — and a good reminder to spot-check how the phonemizer handles proper nouns.
• Slight texture softness vs. a large vocoder remains at close listening.
• English only. Everything upstream says about language coverage and biases applies.
• Not a cloning tool. These are synthetic personas built from a teacher we're entitled to use. Don't point this pipeline at a real person's voice without their explicit consent.

The family

Model	What it is
scrappy-voice-family-1	← you are here. All four voices + blending, one checkpoint
scrappy-voice-1	Scrappy, solo — the original, and this model's warm start
clara-voice-1	Clara, solo
silas-voice-1	Silas, solo
pip-voice-1	Pip, solo

Use the family model unless you specifically want one voice at its sharpest, or want the smaller single-voice checkpoint.

Train your own family (trainer/)

The trainer/ directory holds the multi-speaker version of the stack upstream deliberately omits:

• prep_filelists_ms.py — phonemizes transcripts with the model's own frontend, validates every symbol against the release inventory, and emits speaker-id-tagged filelists plus the speakers.json name→id map.
• train_ms.py — the full loop with speaker conditioning: VITS losses, torchaudio mel transforms (slaney/slaney — no librosa dependency), warm-start loading that grows a single-speaker checkpoint into a multi-speaker one, and drop-in candidate export.
• eval_candidate_ms.py — renders fixed prompts from any candidate, per voice, for A/B listening.

You also need the cython monotonic-alignment kernel from the canonical VITS repo (the release stubs it out) — build it and drop the package into runtime/. Data contract per voice: mono 24kHz clips, verified transcripts, 1–5+ hours. Gate every corpus with ASR round-trips, and warm-start from a voice you've already distilled rather than the stock base — we measured that, and it wins.

Provenance & takedown

All training audio was synthesized: three of the four voices from written descriptions via a voice-design model, the fourth from a commercial cloud narrator voice. Synthetic personas throughout — no real person's voice was cloned. If you're a rights holder with a concern, open a discussion on this repo and we'll respond promptly.

Credits

• owensong/Inflect-Micro-v2 — base model, runtime, and an unusually honest set of docs (Apache-2.0)
• VITS (MIT) — architecture lineage + alignment kernel
• The Hugging Face team — built and gifted us the first demo Space on free ZeroGPU 🤗
• Built by ScrappyLabs — we do this kind of thing to stay sharp. Bring your own AI; we keep it wrangled.

Trained with: scrappylabsai/inflect-trainer — the fine-tuning stack (single-voice, multi-speaker, and the corpus QC gate), open source.