Audio-Text-to-Text
Transformers
Safetensors
English
ultravox
feature-extraction
audio
speech
speech-language-model
qwen
telephony
voice-agent
automatic-speech-recognition
custom_code
Instructions to use QuantumDesk-AI/ultravox-qwen3.6-27b-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use QuantumDesk-AI/ultravox-qwen3.6-27b-v2 with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("QuantumDesk-AI/ultravox-qwen3.6-27b-v2", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
docs: cross-link base<->v2, add known limitations, fix stale benchmark promise, enrich v2 metadata
Browse files
README.md
CHANGED
|
@@ -4,12 +4,18 @@ language:
|
|
| 4 |
- en
|
| 5 |
library_name: transformers
|
| 6 |
pipeline_tag: audio-text-to-text
|
| 7 |
-
base_model:
|
|
|
|
|
|
|
| 8 |
tags:
|
| 9 |
-
-
|
| 10 |
- speech
|
|
|
|
|
|
|
|
|
|
| 11 |
- telephony
|
| 12 |
- voice-agent
|
|
|
|
| 13 |
---
|
| 14 |
|
| 15 |
# ultravox-qwen3.6-27b-v2
|
|
@@ -21,7 +27,9 @@ directly through a Whisper-large-v3-turbo encoder + projection adapter (LoRA r=1
|
|
| 21 |
encoder; knowledge-distillation training against the text LLM as teacher).
|
| 22 |
|
| 23 |
Trained by **QuantumDesk** for receptionist/sales phone agents: heavy real-telephony and
|
| 24 |
-
entity-dense data rather than multilingual benchmark balance.
|
|
|
|
|
|
|
| 25 |
|
| 26 |
## Benchmarks (native inference, greedy, n=100 per set)
|
| 27 |
|
|
@@ -73,4 +81,17 @@ currently loses some accuracy under vLLM relative to native transformers inferen
|
|
| 73 |
maximum quality use HF transformers, or vLLM with `--no-enable-chunked-prefill` and
|
| 74 |
default (fp32) mamba cache dtypes.
|
| 75 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 76 |
Apache 2.0. Built with the Ultravox training framework by Fixie.ai.
|
|
|
|
| 4 |
- en
|
| 5 |
library_name: transformers
|
| 6 |
pipeline_tag: audio-text-to-text
|
| 7 |
+
base_model:
|
| 8 |
+
- Qwen/Qwen3.6-27B
|
| 9 |
+
- openai/whisper-large-v3-turbo
|
| 10 |
tags:
|
| 11 |
+
- audio
|
| 12 |
- speech
|
| 13 |
+
- speech-language-model
|
| 14 |
+
- ultravox
|
| 15 |
+
- qwen
|
| 16 |
- telephony
|
| 17 |
- voice-agent
|
| 18 |
+
- automatic-speech-recognition
|
| 19 |
---
|
| 20 |
|
| 21 |
# ultravox-qwen3.6-27b-v2
|
|
|
|
| 27 |
encoder; knowledge-distillation training against the text LLM as teacher).
|
| 28 |
|
| 29 |
Trained by **QuantumDesk** for receptionist/sales phone agents: heavy real-telephony and
|
| 30 |
+
entity-dense data rather than multilingual benchmark balance. This is the **production
|
| 31 |
+
successor** to [ultravox-qwen3.6-27b-base](https://huggingface.co/QuantumDesk-AI/ultravox-qwen3.6-27b-base)
|
| 32 |
+
(the early English-only research checkpoint) — use this one for real voice-agent work.
|
| 33 |
|
| 34 |
## Benchmarks (native inference, greedy, n=100 per set)
|
| 35 |
|
|
|
|
| 81 |
maximum quality use HF transformers, or vLLM with `--no-enable-chunked-prefill` and
|
| 82 |
default (fp32) mamba cache dtypes.
|
| 83 |
|
| 84 |
+
## Known limitations
|
| 85 |
+
|
| 86 |
+
- **Short utterances degrade sharply.** The benchmarks above are sentence-length inputs.
|
| 87 |
+
On isolated sub-second turns — "Yes", "Hello", "Okay", single-word backchannels — the
|
| 88 |
+
model frequently mistranscribes or hallucinates (observed in live deployment; the
|
| 89 |
+
training mix is dominated by sentence-length corpora). If you use this in a live
|
| 90 |
+
agent, buffer very short VAD segments with surrounding context, or route turns under
|
| 91 |
+
~1.5 s through a fallback. A v3 addressing this with short-turn training data is in
|
| 92 |
+
progress.
|
| 93 |
+
- **English only** — degraded quality on other languages and heavily accented English.
|
| 94 |
+
- **Response style is trained to the prompt shape** shown above; deviating from it
|
| 95 |
+
(especially for conversational use) reduces quality.
|
| 96 |
+
|
| 97 |
Apache 2.0. Built with the Ultravox training framework by Fixie.ai.
|