Ultravox v0.5 Llama 3.2 1B (Core ML Native)
Private Core ML release of Ultravox v0.5 (fixie-ai/ultravox-v0_5-llama-3_2-1b) for TranslateBlue on iOS and macOS.
Architecture
- Audio Encoder: Whisper-large-v3-turbo encoder (128 mel bins)
- Projector: SwiGLU 2-layer MLP (1280 → 2048)
- LLM Decoder: Llama-3.2-1B-Instruct with stateful KV cache (
UltravoxLlamaKVCache.mlpackage)
Artifacts
| Path | Role | Notes |
|---|---|---|
AudioEncoder.mlpackage |
Encoder source | On-device compile fallback (macOS only) |
AudioEncoder.mlmodelc |
Encoder (macOS) | Do not load on iOS |
AudioEncoder-ios.mlmodelc |
Encoder (iOS) | Pre-compiled; preferred on iPhone/iPad |
UltravoxLlamaKVCache.mlpackage |
Decoder | Stateful Core ML; 2.47 GB weight.bin |
tokenizer.json |
BPE tokenizer | Legacy string merges (280,147 entries); required for inference |
tokenizer_config.json |
Tokenizer metadata | Not sufficient alone — tokenizer.json is required |
ultravox_llama_kvcache_manifest.json |
Decoder manifest | Input/output/state names |
Platform-specific compiled encoder bundles are also published in sibling repos:
Requirements
- iOS 18.0+ or macOS 15.0+
- Physical device required for end-to-end validation — the decoder is a stateful Core ML model (
computeUnits = .cpuAndGPU; ANE unsupported today).
iOS Simulator is not supported
UltravoxLlamaKVCache.mlpackage fails to compile on the iOS Simulator. Observed error:
Espresso exception: "Invalid state": MpsGraph backend validation on incompatible OS
compilation failed — Failed to build the model execution plan … model.mil … error code: -14
Use a physical iPhone or iPad for decoder load and speech-to-text translation tests.
Tokenizer format
tokenizer.json uses the legacy BPE merges string format (["Ġ Ġ", …]), not the tokenizers 0.20+ pair format ([["Ġ","Ġ"], …]). Some Rust tokenizers loaders silently return null on the new format. Byte-level BPE encodes spaces as Ġ, so the string join is unambiguous and reversible.
Tasks
- Direct Speech-to-Text Translation (S2TT)
- Context-Aware Speech Recognition (ASR)
- Voice Conversation / QA
- Downloads last month
- -