--- license: apache-2.0 base_model: OpenMOSS-Team/MOSS-TTS-Local-Transformer pipeline_tag: text-to-speech tags: - text-to-speech - tts - moss-tts - gguf - llama.cpp - q4_k_m - reference-conditioned language: - en --- ![Roo Voice](https://design8.b-cdn.net/huggingface/roo-voice-transformers.png) ![AMD](https://img.shields.io/badge/AMD-Vulkan_/_ROCm-ED1C24?style=for-the-badge&logo=amd&logoColor=white) ![NVIDIA CUDA](https://img.shields.io/badge/NVIDIA-CUDA-76B900?style=for-the-badge&logo=nvidia&logoColor=white) ![CPU](https://img.shields.io/badge/CPU-llama.cpp-555?style=for-the-badge) # Roo-Voice · MOSS-TTS-Local-Transformer · GGUF (llama.cpp) Roo's voice — that signature baritone with the estuary accent that stands the hair up on the back of your neck — as a **GGUF** model for **llama.cpp**. This is the **cross-vendor** build: one file that runs on **AMD, NVIDIA, and CPU** (Vulkan / ROCm / CUDA / Metal), because llama.cpp is portable across all of them. Load it, and Roo can whisper to you all day long. A GGUF conversion of a **full-model supervised fine-tune** of [MOSS-TTS-Local-Transformer](https://huggingface.co/OpenMOSS-Team/MOSS-TTS-Local-Transformer), trained on Roo's own recordings and specialised on his single voice. ## ⚠️ What actually makes the voice — read first This is a **reference-conditioned** model, and both halves matter: - **The fine-tune is what makes it Roo.** The base model has never heard this speaker — a base model plus any reference clip will *not* give you Roo's baritone or his estuary accent. That voice lives in the weights, put there by the supervised fine-tune on his recordings. - **The reference completes the delivery.** `reference_24k.wav` (bundled) conditions the fine-tuned model at inference and is **required** to produce the voice. ## Files | File | What | Size | |---|---|---| | `moss-tts-local-roo-ip172-Q4_K_M.gguf` | **4-bit** (recommended) — smallest, GPU-friendly | ~1.76 GB | | `moss-tts-local-roo-ip172-BF16.gguf` | Full-precision GGUF (largest quality headroom) | ~5.5 GB | | `reference_24k.wav` | The identity reference (**required**) | — | ## Why GGUF here — the cross-vendor lane The same portable model runs on both GPU vendors; only a couple of formats are locked to one: | Build | AMD | NVIDIA | Apple | |---|---|---|---| | **This GGUF** (llama.cpp) | ✅ Vulkan / ROCm | ✅ CUDA / Vulkan | ✅ Metal | | [MLX 4-bit / 8-bit](https://huggingface.co/abliter8-ai/Roo-Voice_MOSS_TTS_LT_mlx4) | — | — | ✅ only | | [int4 / int8](https://huggingface.co/abliter8-ai/Roo-Voice_MOSS_TTS_LT_int4) (bitsandbytes) | ⚠️ experimental ROCm | ✅ | — | **If you're on AMD Radeon, this GGUF is your path** — `bitsandbytes` (the int4/int8 builds) is NVIDIA-first and unreliable on consumer Radeon, so use llama.cpp's Vulkan or ROCm backend here instead. > **AMD support is experimental / best-effort.** This build is validated on **NVIDIA CUDA**. On AMD, > a **desktop-class Radeon (RDNA3/4 dGPU) with ROCm** is expected to accelerate well, but we could not > fully validate it on our own hardware. Note: on a low-power **iGPU** (e.g. Radeon 890M) the current > MOSS-TTS llama.cpp fork's custom ops did **not** offload to the GPU under Vulkan in our testing and > ran CPU-bound — so treat iGPU performance as CPU-class for now. Contributions of Vulkan/ROCm kernels > for the MOSS-TTS ops would improve this. For AMD, **SGLang-Omni / vLLM-Omni (ROCm)** are also worth > trying for the transformers builds. ## How to run it MOSS-TTS Local is a two-part system: an **LM backbone** (this GGUF, run by llama.cpp) and a neural **audio codec** (the MOSS-Audio-Tokenizer, run via ONNX Runtime) that turns the model's tokens into 24 kHz audio and encodes the reference. You need the OpenMOSS llama.cpp integration, which adds the `llama-moss-tts` tool and the delay-pattern / 32-codebook decoding: 1. **Build the fork** — OpenMOSS's MOSS-TTS llama.cpp integration ([PR](https://github.com/OpenMOSS/llama.cpp/pull/6)). Build with your GPU backend: `-DGGML_VULKAN=ON` (portable, best on AMD), `-DGGML_CUDA=ON` (NVIDIA), or `-DGGML_HIP=ON` (AMD ROCm). 2. **Get the codec** — `OpenMOSS-Team/MOSS-Audio-Tokenizer` (Apache-2.0). Export or fetch its ONNX encoder/decoder for ONNX Runtime. Install `onnxruntime` (CPU) or the GPU package matching your CUDA/ROCm for fast decode. 3. **Build the generation reference** from `reference_24k.wav` + your text (`tools/tts/moss-tts-build-generation-ref.py`), then run `llama-moss-tts` with `--n-gpu-layers -1` and point `--audio-decoder-onnx` at the codec. It emits a 24 kHz WAV. > **Note on speed:** put the codec on the GPU (an onnxruntime GPU EP, or DirectML on Windows/AMD). The > LM backbone is fast on GPU; if the *codec* decode runs on CPU it becomes the bottleneck. Decoding contract for this voice: seed 42, temperature 1.0, top-k 50, top-p 0.95, repetition penalty 1.1, 32 RVQ codebooks, 24 kHz. ## Prefer a turnkey path? If you don't want to build the fork: on **Apple Silicon** use the [MLX builds](https://huggingface.co/abliter8-ai/Roo-Voice_MOSS_TTS_LT_mlx4); on **NVIDIA** use the [int4 / int8](https://huggingface.co/abliter8-ai/Roo-Voice_MOSS_TTS_LT_int4) transformers builds. The GGUF is the one to reach for when you want **AMD** or a single cross-vendor file. ## Limitations - **Reference-conditioned** — the bundled `reference_24k.wav` must ride along; there is no text-only path. - Single voice by design (this is Roo, not a multi-speaker system). - Requires the OpenMOSS MOSS-TTS llama.cpp integration + the ONNX audio codec — it is not a drop-in `llama-cli` model. ## Provenance & license Quantized/exported form of an accepted single-speaker MOSS-TTS Local supervised fine-tune. The base model and audio codec are Apache-2.0 (OpenMOSS); weights derived from them are redistributed here under the same license.