---
license: apache-2.0
library_name: mlx
pipeline_tag: automatic-speech-recognition
base_model: ibm-granite/granite-speech-4.1-2b
base_model_relation: quantized
language:
  - multilingual
  - en
  - fr
  - de
  - es
  - pt
  - ja
tags:
  - mlx
  - mlx-audio
  - automatic-speech-recognition
  - speech-to-text
  - speech-translation
  - granite
  - granite-speech
  - autoregressive
  - quantized
  - 5-bit
---

# Granite Speech 4.1 2B MLX 5-bit

Quality-oriented mixed-precision MLX conversion of
[`ibm-granite/granite-speech-4.1-2b`](https://huggingface.co/ibm-granite/granite-speech-4.1-2b)
for Apple Silicon. This is the **autoregressive Granite Speech model**, not the
separate NAR variant.

The model runs with
[`mlx-audio`](https://github.com/Blaizzy/mlx-audio) and supports multilingual
automatic speech recognition (ASR), speech translation (AST), punctuation and
capitalization, and keyword-biased transcription.

## Quantization

This conversion uses post-training weight quantization only. No training,
fine-tuning, calibration dataset, or importance matrix was used.

| Component | Precision |
|---|---|
| 16-layer Conformer speech encoder | BF16 |
| 2-layer Q-Former speech projector | BF16 |
| Granite language model, ordinary eligible internal linear layers | MLX affine 5-bit, group size 64 |
| All `v_proj` and `down_proj` layers | MLX affine 6-bit, group size 64 |
| Token embedding | BF16 |
| Language-model output head | BF16 |
| Norms, biases, and unsupported tensors | BF16 |

Value and down projections receive extra precision because mixed-tensor
quantizers commonly protect these sensitive paths. The acoustic stack, token
embedding, and output head remain BF16. This is conceptually similar to medium
mixed-tensor quantizations such as GGUF Q5_K_M, but the file format and
numerical representation are MLX affine quantization rather than GGUF K-quants.

- Source revision: `de575db64086f84fdc79da4932d1076e965bc546`
- Effective average reported by MLX: 9.691 bits per weight
- `model.safetensors`: approximately 2.5 GB

## Requirements

- Apple Silicon Mac
- macOS 14 or later
- `mlx-audio >= 0.4.5`

```bash
pip install -U mlx-audio
```

## Usage

```bash
python -m mlx_audio.stt.generate \
  --model divydeep/granite-speech-4.1-2b-mlx-5bit \
  --audio audio.wav \
  --output-path transcript \
  --format txt \
  --prompt "transcribe the speech with proper punctuation and capitalization."
```

Python:

```python
from mlx_audio.stt.utils import load_model

model = load_model("divydeep/granite-speech-4.1-2b-mlx-5bit")
result = model.generate(
    "audio.wav",
    prompt="transcribe the speech with proper punctuation and capitalization.",
    temperature=0.0,
)
print(result.text)
```

Greedy decoding with `temperature=0.0` is recommended for transcription.

## Prompts

| Task | Prompt |
|---|---|
| Raw ASR | `can you transcribe the speech into a written format?` |
| Punctuated ASR | `transcribe the speech with proper punctuation and capitalization.` |
| Keyword-biased ASR | `transcribe the speech to text. Keywords: <kw1>, <kw2>, ...` |
| Speech translation | `translate the speech to <language>.` |
| Punctuated translation | `translate the speech to <language> with proper punctuation and capitalization.` |

For non-English punctuation, translation, and keyword biasing, use the English
prompt forms recommended by IBM.

## Validation

The conversion was validated by:

- Strictly loading all weights with `mlx-audio`.
- Confirming the architecture is `granite_speech`, not `granite_speech_nar`.
- Running greedy transcription on IBM's bundled `multilingual_sample.wav`.
- Checking preservation of English and French text, punctuation, accents, and
  capitalization in the generated transcript.

This is a conversion smoke test, not a full WER benchmark. The upstream IBM
evaluation results describe the original model and should not be interpreted as
measured results for this quantized conversion.

## Limitations

- Quantization may affect rare words, names, punctuation, casing, keyword
  biasing, translation, and difficult or noisy audio.
- This model has not been independently evaluated on the complete IBM benchmark
  suite.
- The model is autoregressive and does not provide the throughput behavior of
  the separate NAR architecture.
- Refer to the upstream model card for intended use, language coverage, safety
  considerations, training data, and architectural details.

## References

- [Original model and authoritative model card](https://huggingface.co/ibm-granite/granite-speech-4.1-2b)
- [Granite Speech paper](https://arxiv.org/abs/2505.08699)
- [MLX Audio](https://github.com/Blaizzy/mlx-audio)

## License

Apache-2.0, matching the original model.

## Citation

```bibtex
@misc{granite-speech-4.1-2b,
  title={Granite 4.1 Speech},
  author={IBM Granite Speech Team},
  year={2026},
  url={https://huggingface.co/ibm-granite/granite-speech-4.1-2b}
}
```