---
license: apache-2.0
language:
  - en
library_name: transformers
pipeline_tag: audio-text-to-text
base_model: Qwen/Qwen3.6-27B
tags:
  - ultravox
  - speech
  - telephony
  - voice-agent
---

# ultravox-qwen3.6-27b-v2

A speech-language model for **phone-call voice agents**, built on the
[Ultravox](https://github.com/fixie-ai/ultravox) architecture: a frozen
[Qwen 3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) LLM that consumes audio
directly through a Whisper-large-v3-turbo encoder + projection adapter (LoRA r=16 on the
encoder; knowledge-distillation training against the text LLM as teacher).

Trained by **QuantumDesk** for receptionist/sales phone agents: heavy real-telephony and
entity-dense data rather than multilingual benchmark balance.

## Benchmarks (native inference, greedy, n=100 per set)

| Metric | v2 | ultravox-v0_6-gemma-3-27b (same rig) |
|---|---|---|
| LibriSpeech test-clean WER | **2.08%** | 1.90% |
| Switchboard (real phone calls) WER | **9.67%** | 21.2% |
| Phone-number digit accuracy (phone-quality audio) | **97.0%** | 98.5% |
| Person-name accuracy | **78.8%** | 60.6% |
| Spelled-email accuracy | **53.2%** | 50.6% |
| Business-name accuracy | **91.2%** | 73.5% |

On conversational phone audio — the target domain — this model roughly **halves the word
error rate** of general-purpose Ultravox models, while matching digit accuracy and
substantially exceeding name recognition.

## Conversational use

v2 responds natively to audio (not transcription-only). It is trained on this exact user
turn shape:

```
[
  {"role": "system", "content": "<your agent persona>"},
  {"role": "user", "content": [
    {"type": "text", "text": "Respond conversationally to what the user just said. Keep replies short — one or two natural sentences. Never use markdown, code, symbols, or headers. Never invent quotes. If you didn't catch what they said, ask them to repeat. "},
    {"type": "input_audio", "input_audio": {"data": "<b64 wav>", "format": "wav"}}
  ]}
]
```

For transcription use: `"Repeat the following text, without any explanation: "` + audio.
A persona system prompt is strongly recommended for conversational use.

## Training

Two phases from scratch on 2×H200:
1. **Phase 1B** (~35K steps, eff. batch 16): English ASR corpora + real telephone
   conversations, with telephony augmentation (8kHz resample, AMR/μ-law compression,
   band-pass, pink noise, gain).
2. **Phase 2** (15K steps): 150K synthetic entity-dense utterances (phone numbers, names,
   spelled emails, addresses, dates/times across 31 TTS voices) + 40K spoken-dialogue
   response pairs + anti-forgetting garnish from phase-1 corpora.

## Serving

Works with vLLM (`--trust-remote-code`). Note: Qwen 3.6's hybrid linear attention
currently loses some accuracy under vLLM relative to native transformers inference; for
maximum quality use HF transformers, or vLLM with `--no-enable-chunked-prefill` and
default (fp32) mamba cache dtypes.

Apache 2.0. Built with the Ultravox training framework by Fixie.ai.