---
base_model: mistralai/Voxtral-Mini-4B-Realtime-2602
language:
  - ar
  - de
  - en
  - es
  - fr
  - hi
  - it
  - nl
  - pt
  - zh
  - ja
  - ko
  - ru
library_name: mlx
license: apache-2.0
pipeline_tag: automatic-speech-recognition
tags:
  - mlx
  - mlx-audio
  - speech-to-text
  - streaming
  - realtime
  - voxtral
  - fp16
---

# Voxtral Mini 4B Realtime 4bit (float16)

This is a **4-bit quantized, float16-base** [MLX](https://github.com/ml-explore/mlx) conversion of [mistralai/Voxtral-Mini-4B-Realtime-2602](https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-2602).

## Which variant should you pick?

| Chip           | Recommended                                                                                            | Why                                                                                                                      |
|----------------|--------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------|
| **M1 / M2**    | **This repo (`-4bit-fp16`)**                                                                           | Metal on M1/M2 has no native `bfloat16` ALU; bf16 ops fall back to a slower path. `float16` stays on the fast GPU path.  |
| **M3 / M4+**   | [`iris-sfg/Voxtral-Mini-4B-Realtime-2602-4bit`](https://huggingface.co/iris-sfg/Voxtral-Mini-4B-Realtime-2602-4bit) (bf16) | bf16 is natively supported and gives the same speed as fp16 with a wider dynamic range (slightly safer numerics).       |

Only the non-quantized weights differ between the two repos (norms, biases, scales, some embeddings). The quantized mat-mul weights are bit-identical. Transcription output is byte-identical on a 20 s French clip at temperature 0 (verified locally).

## Conversion

Source model:

- `mistralai/Voxtral-Mini-4B-Realtime-2602`

Local conversion command:

```bash
python -m mlx_audio.convert \
  --hf-path mistralai/Voxtral-Mini-4B-Realtime-2602 \
  --mlx-path /path/to/Voxtral-Mini-4B-Realtime-2602-4bit-fp16 \
  --quantize \
  --q-group-size 64 \
  --q-bits 4 \
  --dtype float16 \
  --model-domain stt
```

Quantization config:

- bits: `4`
- group size: `64`
- mode: `affine`
- non-quant dtype: `float16`

## Files

Only the MLX runtime artifacts needed for inference:

- `model.safetensors`
- `model.safetensors.index.json`
- `config.json`
- `generation_config.json`
- `params.json`
- `processor_config.json`
- `tekken.json`

## Usage

```bash
pip install "mlx-audio[stt]"
```

```python
from mlx_audio.stt.utils import load_model

model = load_model("iris-sfg/Voxtral-Mini-4B-Realtime-2602-4bit-fp16")
result = model.generate("audio.wav")
print(result.text)
```

## Notes

- Base model license remains Apache 2.0.
- On M3/M4, prefer the `-4bit` (bf16) repo; there is no speed benefit to fp16 there and bf16's wider exponent range is slightly more robust.
- Transcription quality was verified identical to the bf16 variant at `temperature=0` on a 20 s French parliamentary audio clip.