Finetuned VoxCPM1.5

This repo contains the base model and LoRA adapters for VoxCPM 1.5, trained on this Orpheus 3B TTS distillation dataset.

Voices

Voice	Dataset Name	LoRA Path
Female	tara	`lora/female/`
Male	zac	`lora/male/`

Training Configuration

Fine-tuned using LoRA (Low-Rank Adaptation) applied to both the language model and diffusion transformer components. The projection layers between encoder, LM, and DiT are also adapted to better capture voice-specific acoustic characteristics.

LoRA Parameters

Parameter	Value
Rank (`r`)	48
Alpha	48
Dropout	0.05
Enable LM LoRA	True
Enable DiT LoRA	True
Enable Projection LoRA	True
Target Modules (LM)	`q_proj`, `v_proj`, `k_proj`, `o_proj`
Target Modules (DiT)	`q_proj`, `v_proj`, `k_proj`, `o_proj`

Training Hyperparameters

Training was performed on the full dataset without a validation split to maximize data utilization. A cosine learning rate schedule with warmup ensures stable convergence.

Parameter	Value
Iterations	4000
Batch Size	16
Learning Rate	1e-4
Warmup Steps	200
Weight Decay	0.01
Scheduler	Cosine with warmup
Validation Split	0.0
Precision	bfloat16

Sample Generation Configuration

Voice samples were generated using lower CFG values for more expressive, natural-sounding output. Higher inference timesteps will further improve audio quality during inference (compared to sample quality) at the expense of speed.

Parameter	Value
CFG Value	1.5
Inference Timesteps	25
Max Length	1000
Precision	bfloat16

Files

File	Description
`lora/<voice>/lora_weights.safetensors`	LoRA adapter weights for each voice
`lora/<voice>/lora_config.json`	LoRA configuration and base model reference
`model.safetensors`	Base model weights
`config.json`	Model configuration
`audiovae.pth`	Audio VAE weights
`tokenizer.json`	Tokenizer
`tokenizer_config.json`	Tokenizer configuration
`samples/`	Voice samples for each voice

License

This model inherits the Apache 2.0 license from the base model.

Citation

If you use this model, please cite the original VoxCPM paper:

@article{voxcpm2025,
  title        = {VoxCPM: Tokenizer-Free TTS for Context-Aware Speech Generation and True-to-Life Voice Cloning},
  author       = {Zhou, Yixuan and Zeng, Guoyang and Liu, Xin and Li, Xiang and Yu, Renjie and Wang, Ziyang and Ye, Runchuan and Sun, Weiyue and Gui, Jiancheng and Li, Kehan and Wu, Zhiyong and Liu, Zhiyuan},
  journal      = {arXiv preprint arXiv:2509.24650},
  year         = {2025},
}