| --- |
| license: apache-2.0 |
| base_model: OpenMOSS-Team/MOSS-TTS-Local-Transformer |
| pipeline_tag: text-to-speech |
| tags: |
| - text-to-speech |
| - tts |
| - moss-tts |
| - gguf |
| - llama.cpp |
| - q4_k_m |
| - reference-conditioned |
| language: |
| - en |
| --- |
| |
|  |
|
|
|  |
|  |
|  |
|
|
| # Roo-Voice · MOSS-TTS-Local-Transformer · GGUF (llama.cpp) |
|
|
| Roo's voice — that signature baritone with the estuary accent that stands the hair up on the back of |
| your neck — as a **GGUF** model for **llama.cpp**. This is the **cross-vendor** build: one file that |
| runs on **AMD, NVIDIA, and CPU** (Vulkan / ROCm / CUDA / Metal), because llama.cpp is portable across |
| all of them. Load it, and Roo can whisper to you all day long. |
|
|
| A GGUF conversion of a **full-model supervised fine-tune** of |
| [MOSS-TTS-Local-Transformer](https://huggingface.co/OpenMOSS-Team/MOSS-TTS-Local-Transformer), |
| trained on Roo's own recordings and specialised on his single voice. |
|
|
| ## ⚠️ What actually makes the voice — read first |
|
|
| This is a **reference-conditioned** model, and both halves matter: |
|
|
| - **The fine-tune is what makes it Roo.** The base model has never heard this speaker — a base model |
| plus any reference clip will *not* give you Roo's baritone or his estuary accent. That voice lives |
| in the weights, put there by the supervised fine-tune on his recordings. |
| - **The reference completes the delivery.** `reference_24k.wav` (bundled) conditions the fine-tuned |
| model at inference and is **required** to produce the voice. |
|
|
| ## Files |
|
|
| | File | What | Size | |
| |---|---|---| |
| | `moss-tts-local-roo-ip172-Q4_K_M.gguf` | **4-bit** (recommended) — smallest, GPU-friendly | ~1.76 GB | |
| | `moss-tts-local-roo-ip172-BF16.gguf` | Full-precision GGUF (largest quality headroom) | ~5.5 GB | |
| | `reference_24k.wav` | The identity reference (**required**) | — | |
|
|
| ## Why GGUF here — the cross-vendor lane |
|
|
| The same portable model runs on both GPU vendors; only a couple of formats are locked to one: |
|
|
| | Build | AMD | NVIDIA | Apple | |
| |---|---|---|---| |
| | **This GGUF** (llama.cpp) | ✅ Vulkan / ROCm | ✅ CUDA / Vulkan | ✅ Metal | |
| | [MLX 4-bit / 8-bit](https://huggingface.co/abliter8-ai/Roo-Voice_MOSS_TTS_LT_mlx4) | — | — | ✅ only | |
| | [int4 / int8](https://huggingface.co/abliter8-ai/Roo-Voice_MOSS_TTS_LT_int4) (bitsandbytes) | ⚠️ experimental ROCm | ✅ | — | |
|
|
| **If you're on AMD Radeon, this GGUF is your path** — `bitsandbytes` (the int4/int8 builds) is |
| NVIDIA-first and unreliable on consumer Radeon, so use llama.cpp's Vulkan or ROCm backend here instead. |
|
|
| > **AMD support is experimental / best-effort.** This build is validated on **NVIDIA CUDA**. On AMD, |
| > a **desktop-class Radeon (RDNA3/4 dGPU) with ROCm** is expected to accelerate well, but we could not |
| > fully validate it on our own hardware. Note: on a low-power **iGPU** (e.g. Radeon 890M) the current |
| > MOSS-TTS llama.cpp fork's custom ops did **not** offload to the GPU under Vulkan in our testing and |
| > ran CPU-bound — so treat iGPU performance as CPU-class for now. Contributions of Vulkan/ROCm kernels |
| > for the MOSS-TTS ops would improve this. For AMD, **SGLang-Omni / vLLM-Omni (ROCm)** are also worth |
| > trying for the transformers builds. |
|
|
| ## How to run it |
|
|
| MOSS-TTS Local is a two-part system: an **LM backbone** (this GGUF, run by llama.cpp) and a neural |
| **audio codec** (the MOSS-Audio-Tokenizer, run via ONNX Runtime) that turns the model's tokens into |
| 24 kHz audio and encodes the reference. You need the OpenMOSS llama.cpp integration, which adds the |
| `llama-moss-tts` tool and the delay-pattern / 32-codebook decoding: |
|
|
| 1. **Build the fork** — OpenMOSS's MOSS-TTS llama.cpp integration |
| ([PR](https://github.com/OpenMOSS/llama.cpp/pull/6)). Build with your GPU backend: |
| `-DGGML_VULKAN=ON` (portable, best on AMD), `-DGGML_CUDA=ON` (NVIDIA), or `-DGGML_HIP=ON` (AMD ROCm). |
| 2. **Get the codec** — `OpenMOSS-Team/MOSS-Audio-Tokenizer` (Apache-2.0). Export or fetch its ONNX |
| encoder/decoder for ONNX Runtime. Install `onnxruntime` (CPU) or the GPU package matching your |
| CUDA/ROCm for fast decode. |
| 3. **Build the generation reference** from `reference_24k.wav` + your text |
| (`tools/tts/moss-tts-build-generation-ref.py`), then run `llama-moss-tts` with `--n-gpu-layers -1` |
| and point `--audio-decoder-onnx` at the codec. It emits a 24 kHz WAV. |
|
|
| > **Note on speed:** put the codec on the GPU (an onnxruntime GPU EP, or DirectML on Windows/AMD). The |
| > LM backbone is fast on GPU; if the *codec* decode runs on CPU it becomes the bottleneck. |
|
|
| Decoding contract for this voice: seed 42, temperature 1.0, top-k 50, top-p 0.95, repetition |
| penalty 1.1, 32 RVQ codebooks, 24 kHz. |
|
|
| ## Prefer a turnkey path? |
|
|
| If you don't want to build the fork: on **Apple Silicon** use the |
| [MLX builds](https://huggingface.co/abliter8-ai/Roo-Voice_MOSS_TTS_LT_mlx4); on **NVIDIA** use the |
| [int4 / int8](https://huggingface.co/abliter8-ai/Roo-Voice_MOSS_TTS_LT_int4) transformers builds. The |
| GGUF is the one to reach for when you want **AMD** or a single cross-vendor file. |
|
|
| ## Limitations |
|
|
| - **Reference-conditioned** — the bundled `reference_24k.wav` must ride along; there is no text-only path. |
| - Single voice by design (this is Roo, not a multi-speaker system). |
| - Requires the OpenMOSS MOSS-TTS llama.cpp integration + the ONNX audio codec — it is not a |
| drop-in `llama-cli` model. |
|
|
| ## Provenance & license |
|
|
| Quantized/exported form of an accepted single-speaker MOSS-TTS Local supervised fine-tune. The base |
| model and audio codec are Apache-2.0 (OpenMOSS); weights derived from them are redistributed here |
| under the same license. |
|
|
