Qwen3-TTS-12Hz-0.6B-Base · Hindi LoRA Finetune (BitsAndBytes NF4)

A Hindi (hi) LoRA finetune of Qwen/Qwen3-TTS-12Hz-0.6B-Base, with the LoRA adapter merged into the base weights and the result quantized to 4-bit NF4 using bitsandbytes.

The base Qwen3-TTS family ships with 10 built-in languages (Chinese, English, Japanese, Korean, German, French, Russian, Portuguese, Spanish, Italian) but does not include Hindi. This checkpoint extends the base model toward Hindi speech synthesis and voice cloning while keeping the memory footprint small enough to run on consumer GPUs.

Model Details


Base model	`Qwen/Qwen3-TTS-12Hz-0.6B-Base`
Architecture	`Qwen3TTSForConditionalGeneration` (discrete multi-codebook LM talker + code predictor)
Parameters	~0.6B (talker), plus speech tokenizer
Target language	Hindi (`hi`)
Finetuning method	LoRA (adapter merged into base)
Quantization	bitsandbytes 4-bit NF4, double quantization, `bfloat16` compute dtype
Speech tokenizer	Qwen3-TTS-Tokenizer-12Hz (12.5 Hz frame rate, 24 kHz audio)
Sample rate	24 kHz
License	Apache-2.0 (inherited from base)

The following modules are kept in higher precision (not quantized) for output quality: text_projection, codec_head, code_predictor, speaker_encoder.

Repository Contents

config.json                 # Model + quantization config
generation_config.json      # Default sampling params
model.safetensors           # 4-bit NF4 quantized weights (~1.2 GB)
merges.txt / vocab.json     # Text tokenizer
tokenizer_config.json
preprocessor_config.json
speech_tokenizer/           # Qwen3-TTS-Tokenizer-12Hz (encode/decode audio)

Installation

conda create -n qwen3-tts python=3.12 -y
conda activate qwen3-tts

pip install -U qwen-tts
pip install -U bitsandbytes        # required to load the 4-bit weights
# Optional, for faster inference on supported GPUs:
pip install -U flash-attn --no-build-isolation

Usage

The quantization settings are stored in config.json, so the 4-bit weights load automatically — no extra BitsAndBytesConfig is needed.

Voice cloning (Hindi)

import torch
import soundfile as sf
from qwen_tts import Qwen3TTSModel

MODEL_ID = "<your-username>/Qwen-3-TTS-12Hz-Base-hi-LoRA-Finetuned-BNB-NF4"
# or a local path to this folder

model = Qwen3TTSModel.from_pretrained(
    MODEL_ID,
    device_map="cuda:0",
    attn_implementation="flash_attention_2",  # drop if flash-attn is not installed
)

# Short Hindi reference clip + its transcript for cloning
ref_audio = "path/to/reference_hindi.wav"
ref_text  = "नमस्ते, मेरा नाम आरव है और मुझे संगीत सुनना बहुत पसंद है।"

wavs, sr = model.generate_voice_clone(
    text="आज मौसम बहुत सुहाना है, चलिए थोड़ी देर बाहर टहलने चलते हैं।",
    language="Hindi",
    ref_audio=ref_audio,
    ref_text=ref_text,
)

sf.write("output_hindi_clone.wav", wavs[0], sr)

Loading with Transformers directly

import torch
from transformers import AutoModelForCausalLM, AutoProcessor

model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    device_map="cuda:0",
    trust_remote_code=True,
)
processor = AutoProcessor.from_pretrained(MODEL_ID, trust_remote_code=True)

Finetuning Details


Method	LoRA, adapter merged into base weights
Target language	Hindi (`hi`)
LoRA rank / alpha / dropout	TBD
Target modules	TBD
Training data	TBD
Epochs / steps	TBD
Optimizer / LR / schedule	TBD
Hardware	TBD

Acknowledgements & Citation

This work builds on Qwen3-TTS by the Qwen team at Alibaba Cloud.

@article{Qwen3-TTS,
  title={Qwen3-TTS Technical Report},
  author={Hangrui Hu and Xinfa Zhu and Ting He and Dake Guo and Bin Zhang and Xiong Wang and Zhifang Guo and Ziyue Jiang and Hongkun Hao and Zishan Guo and Xinyu Zhang and Pei Zhang and Baosong Yang and Jin Xu and Jingren Zhou and Junyang Lin},
  journal={arXiv preprint arXiv:2601.15621},
  year={2026}
}