twangodev/cosyvoice3-delta-tts-libritts-585h

CosyVoice3 DELTA-TTS — LibriTTS 585h

Independent DELTA-TTS reproduction for CosyVoice3 0.5B. This is a research adapter, not an official DELTA-TTS or FunAudioLLM release.

The repository contains 93.8M FP32 adapter parameters in safetensors. The pinned CosyVoice3 base model and source checkout are required separately.

Results

Seed-TTS test-en, 1,088 cases, seed 1986, batch size 1, RTX PRO 6000 Blackwell:

System	WER ↓	SIM ↑	Token RTF ↓	End-to-end RTF ↓
CosyVoice3 native AR	2.09%	0.697	—	0.187
This adapter, 16 steps	2.27%	0.683	0.059	0.108

This is a 1.73× end-to-end speedup with a small quality regression. It reproduces the conversion method, not the paper's reported quality improvement. Exact results are in evaluation/.

Usage

This is a custom speech-dllm adapter and is not compatible with transformers.pipeline or the hosted Hugging Face inference widget.

git clone https://github.com/twangodev/speech-dllm.git
git -C speech-dllm checkout a4c423d4a33b620860023168aaa2d9b439e43688

git clone https://github.com/FunAudioLLM/CosyVoice.git /path/to/CosyVoice
git -C /path/to/CosyVoice checkout 074ca6dc9e80a2f424f1f74b48bdd7d3fea531cc

hf download FunAudioLLM/Fun-CosyVoice3-0.5B-2512 \
  --revision 29e01c4e8d000f4bcd70751be16fa94bf3d85a18 \
  --local-dir /path/to/CosyVoice3
hf download twangodev/cosyvoice3-delta-tts-libritts-585h \
  --local-dir /path/to/adapter

cd speech-dllm/environments/cosyvoice
uv sync --frozen
uv run speech-dllm sample /path/to/adapter/inference.yaml \
  --checkpoint /path/to/adapter \
  --source-directory /path/to/CosyVoice \
  --model-directory /path/to/CosyVoice3 \
  --prompt-audio /path/to/prompt.wav \
  --prompt-text "The exact prompt transcript." \
  --text "The text to synthesize." \
  --output sample.wav

The runtime verifies the adapter digest, tensor inventory, and pinned base/source revisions before loading. See adapter_config.json, provenance.json, and validation.json for full details.

Limitations

• English only; one full-suite sampling seed is reported.
• Target duration uses a prompt/text-ratio heuristic and can fail on short prompts or long/out-of-domain text.
• Hard and long-form generation needs more qualification.
• Generation is full-sequence and non-streaming.
• Obtain speaker consent and disclose synthesized speech.

License

Apache-2.0. The required CosyVoice3 base model is distributed separately under Apache-2.0. Training used LibriTTS under CC BY 4.0; see NOTICE.

References: DELTA-TTS, CosyVoice3, and LibriTTS.