{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Qwen3-TTS 0.6B — Indic Multilingual LoRA Inference\n", "\n", "Run Indic-language text-to-speech using **[`aguken-ai/Qwen3-TTS-0.6B-LoRA-Finetuned-Indic-Multilingual`](https://huggingface.co/aguken-ai/Qwen3-TTS-0.6B-LoRA-Finetuned-Indic-Multilingual)** (15 languages × 2 genders) on top of `Qwen/Qwen3-TTS-12Hz-0.6B-Base`.\n", "\n", "1. **Pick a language and gender** in the *Configuration* cell.\n", "2. Run the cells in order.\n", "3. Edit `TEXT` to synthesize anything you want.\n", "\n", "**Hardware:** designed for **Colab free T4** (~3-5 GB VRAM in use). Works on Kaggle T4 / P100 too.\n", "\n", "**Supported languages:** Assamese · Bengali · Bodo · Dogri · Gujarati · Kannada · Konkani · Maithili · Malayalam · Marathi · Nepali · Odia · Punjabi · Tamil · Telugu" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 1. Install dependencies\n", "\n", "Don't downgrade `torch` — use whatever Colab ships. Pinning older `torch` breaks `torchaudio` on Colab's Python 3.12." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "E: Could not open lock file /var/lib/dpkg/lock-frontend - open (13: Permission denied)\n", "E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), are you root?\n", "✅ Dependencies installed\n" ] } ], "source": [ "!apt-get -qq install -y sox libsox-dev ffmpeg libsndfile1 > /dev/null\n", "!pip install -q -U qwen-tts peft transformers soundfile huggingface_hub\n", "print('✅ Dependencies installed')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 2. Imports + environment patches" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/zeus/miniconda3/envs/cloudspace/lib/python3.12/site-packages/torchaudio/_extension/__init__.py:39: UserWarning: TorchAudio native extension could not be loaded (/system/conda/miniconda3/envs/cloudspace/lib/python3.12/site-packages/torchaudio/lib/_torchaudio.abi3.so: undefined symbol: torch_library_impl). Falling back to no-extension mode.\n", " warnings.warn(f\"TorchAudio native extension could not be loaded ({_e}). \"\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "✅ Imports OK\n", "CUDA available: True\n", "Device: NVIDIA L4\n" ] } ], "source": [ "import torch\n", "import soundfile as sf\n", "import IPython.display as ipd\n", "\n", "# PEFT patch: Colab ships torchao 0.10 but PEFT requires >=0.16.\n", "# We don't use torchao here; short-circuit its dispatcher check so PEFT\n", "# falls through to the standard nn.Linear LoRA path.\n", "import peft.tuners.lora.torchao as _torchao_dispatch\n", "_torchao_dispatch.is_torchao_available = lambda: False\n", "\n", "from qwen_tts import Qwen3TTSModel\n", "from peft import PeftModel\n", "from huggingface_hub import hf_hub_download\n", "\n", "print('✅ Imports OK')\n", "print('CUDA available:', torch.cuda.is_available())\n", "if torch.cuda.is_available():\n", " print('Device:', torch.cuda.get_device_name(0))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 3. Configuration — pick your language\n", "\n", "Edit `LANGUAGE` and `GENDER`. Available combos:\n", "\n", "| Language | Code | Folder name |\n", "| --------- | ----- | -------------------------------------------- |\n", "| Assamese | `as` | `assamese_female` / `assamese_male` |\n", "| Bengali | `bn` | `bengali_female` / `bengali_male` |\n", "| Bodo | `brx` | `bodo_female` / `bodo_male` |\n", "| Dogri | `doi` | `dogri_female` / `dogri_male` |\n", "| Gujarati | `gu` | `gujarati_female` / `gujarati_male` |\n", "| Kannada | `kn` | `kannada_female` / `kannada_male` |\n", "| Konkani | `kok` | `konkani_female` / `konkani_male` |\n", "| Maithili | `mai` | `maithili_female` / `maithili_male` |\n", "| Malayalam | `ml` | `malayalam_female` / `malayalam_male` |\n", "| Marathi | `mr` | `marathi_female` / `marathi_male` |\n", "| Nepali | `ne` | `nepali_female` / `nepali_male` |\n", "| Odia | `or` | `odia_female` / `odia_male` |\n", "| Punjabi | `pa` | `punjabi_female` / `punjabi_male` |\n", "| Tamil | `ta` | `tamil_female` / `tamil_male` |\n", "| Telugu | `te` | `telugu_female` / `telugu_male` |" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "✅ Will use adapter: aguken-ai/Qwen3-TTS-0.6B-LoRA-Finetuned-Indic-Multilingual/adapters/telugu_male\n" ] } ], "source": [ "# ─── EDIT THESE ────────────────────────────────────────────\n", "LANGUAGE = 'telugu' # one of the 15 languages above (lowercase)\n", "GENDER = 'male' # 'female' or 'male'\n", "LORA_SCALE = 0.1 # 0.1-0.2 = subtle, 0.3-0.4 = balanced, 0.5 = strong\n", "# ───────────────────────────────────────────────────────────\n", "\n", "# Adapter source — change this if you fork the repo to your own HF org.\n", "BASE_MODEL = 'Qwen/Qwen3-TTS-12Hz-0.6B-Base'\n", "ADAPTER_REPO = 'aguken-ai/Qwen3-TTS-0.6B-LoRA-Finetuned-Indic-Multilingual'\n", "\n", "SUPPORTED_LANGUAGES = {\n", " 'assamese', 'bengali', 'bodo', 'dogri', 'gujarati', 'kannada', 'konkani',\n", " 'maithili', 'malayalam', 'marathi', 'nepali', 'odia', 'punjabi', 'tamil', 'telugu',\n", "}\n", "assert LANGUAGE in SUPPORTED_LANGUAGES, f'LANGUAGE must be one of {sorted(SUPPORTED_LANGUAGES)}'\n", "assert GENDER in ('female', 'male'), \"GENDER must be 'female' or 'male'\"\n", "\n", "ADAPTER_SUB = f'adapters/{LANGUAGE}_{GENDER}'\n", "print(f'✅ Will use adapter: {ADAPTER_REPO}/{ADAPTER_SUB}')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 4. Load base model + LoRA adapter\n", "\n", "First run downloads ~1.8 GB base + ~45 MB adapter (~2 minutes). Subsequent runs use the cache." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "28a2677a9d3a47fe9a6e4908fa4dca14", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Fetching 4 files: 0%| | 0/4 [00:00 11\u001b[0m wavs, sr \u001b[38;5;241m=\u001b[39m \u001b[43mwrapper\u001b[49m\u001b[38;5;241m.\u001b[39mgenerate_voice_clone(\n\u001b[1;32m 12\u001b[0m text\u001b[38;5;241m=\u001b[39mTEXT,\n\u001b[1;32m 13\u001b[0m language\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mauto\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[1;32m 14\u001b[0m ref_audio\u001b[38;5;241m=\u001b[39mref_audio,\n\u001b[1;32m 15\u001b[0m ref_text\u001b[38;5;241m=\u001b[39mref_text,\n\u001b[1;32m 16\u001b[0m )\n\u001b[1;32m 18\u001b[0m sf\u001b[38;5;241m.\u001b[39mwrite(OUTPUT_PATH, wavs[\u001b[38;5;241m0\u001b[39m], sr)\n\u001b[1;32m 19\u001b[0m \u001b[38;5;28mprint\u001b[39m(\u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m✅ Wrote \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mOUTPUT_PATH\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m (\u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mlen\u001b[39m(wavs[\u001b[38;5;241m0\u001b[39m])\u001b[38;5;250m \u001b[39m\u001b[38;5;241m/\u001b[39m\u001b[38;5;250m \u001b[39msr\u001b[38;5;132;01m:\u001b[39;00m\u001b[38;5;124m.1f\u001b[39m\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124ms @ \u001b[39m\u001b[38;5;132;01m{\u001b[39;00msr\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m Hz)\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n", "\u001b[0;31mNameError\u001b[0m: name 'wrapper' is not defined" ] } ], "source": [ "REF_DIR = \"telugu_male\"\n", "ref_audio = f\"{REF_DIR}/ref.wav\"\n", "\n", "with open(f\"{REF_DIR}/ref.txt\", \"r\", encoding=\"utf-8\") as f:\n", " ref_text = f.read().strip()\n", "\n", "TEXT = \"నమస్కారం అందరికీ. నా పేరు **సాయి కృష్ణ**. నేను **హైదరాబాద్** నుండి వచ్చాను. ప్రస్తుతం **ఆగుకెన్ ఏఐ**లో పని చేస్తున్నాను. నాకు కొత్త విషయాలు నేర్చుకోవడం, టెక్నాలజీ గురించి తెలుసుకోవడం, అలాగే నా నైపుణ్యాలను అభివృద్ధి చేసుకోవడం చాలా ఇష్టం. ఈ అవకాశాన్ని ఇచ్చినందుకు ధన్యవాదాలు. మీ అందరితో కలిసి నేర్చుకోవడానికి మరియు నా అనుభవాన్ని పంచుకోవడానికి ఎదురుచూస్తున్నాను. ధన్యవాదాలు!\"\n", "OUTPUT_PATH = f\"output_{LANGUAGE}_{GENDER}.wav\"\n", "\n", "with torch.no_grad():\n", " wavs, sr = wrapper.generate_voice_clone(\n", " text=TEXT,\n", " language=\"auto\",\n", " ref_audio=ref_audio,\n", " ref_text=ref_text,\n", " )\n", "\n", "sf.write(OUTPUT_PATH, wavs[0], sr)\n", "print(f\"✅ Wrote {OUTPUT_PATH} ({len(wavs[0]) / sr:.1f}s @ {sr} Hz)\")\n", "ipd.display(ipd.Audio(OUTPUT_PATH))" ] } ], "metadata": { "accelerator": "GPU", "colab": { "gpuType": "T4", "machine_shape": "hm", "provenance": [] }, "kernelspec": { "display_name": "cloudspace", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.11" } }, "nbformat": 4, "nbformat_minor": 0 }