metadata
base_model: mistralai/Voxtral-Mini-3B-2507
library_name: mlx
license: apache-2.0
pipeline_tag: automatic-speech-recognition
tags:
- voxtral
- audio
- speech
- speech-recognition
- transcription
- translation
- mlx
- rotorquant
- quantization
- 4-bit
Voxtral-Mini-3B-2507-RotorQuant-MLX-4bit
4-bit MLX weight-quantized build of mistralai/Voxtral-Mini-3B-2507 with a RotorQuant KV-cache profile. Recommended streaming-audio default on Apple Silicon.
Hardware compatibility
| Device | VRAM / RAM | Recommendation |
|---|---|---|
| Apple M4 Max 128 GB | ~2.0 GB | recommended — headroom for long context |
| Apple M3 Max 64 GB | ~2.0 GB | comfortable |
| Apple M2 Max 32 GB | ~1.8 GB | fits |
Overview
- Base:
mistralai/Voxtral-Mini-3B-2507— 3B speech-understanding model - Capabilities: transcription, speech translation, audio QA
- Weight precision: 4-bit (group-wise)
- KV-cache profile: RotorQuant (rotational online re-basis)
- Approx. on-disk size: ~1.5 GB
- Runtime: MLX on Apple Silicon
Quickstart
pip install mlx-lm
from mlx_lm import load, generate
model, tokenizer = load("majentik/Voxtral-Mini-3B-2507-RotorQuant-MLX-4bit")
prompt = tokenizer.apply_chat_template(
[{"role": "user", "content": [{"type": "audio", "path": "meeting.wav"},
{"type": "text", "text": "Transcribe."}]}],
add_generation_prompt=True,
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512))
Model specs
| Field | Value |
|---|---|
| Parameters | 3B |
| Weight bits | 4 |
| Group size | 64 |
| Cache profile | RotorQuant |
| Size on disk | ~1.5 GB |
| Target hardware | Apple Silicon (M1/M2/M3/M4) |
| License | Apache 2.0 |
RotorQuant vs TurboQuant
| RotorQuant | TurboQuant | |
|---|---|---|
| Strategy | Rotational online re-basis | Per-head static calibration |
| Memory reduction | ~4x on KV-cache | ~3.5x on KV-cache |
| Best for | Streaming, code-switching | Batch transcription |