---
license: apache-2.0
library_name: mlx
pipeline_tag: automatic-speech-recognition
base_model: ibm-granite/granite-speech-4.1-2b
base_model_relation: quantized
language:
  - multilingual
  - en
  - fr
  - de
  - es
  - pt
  - ja
tags:
  - mlx
  - mlx-audio
  - automatic-speech-recognition
  - speech-to-text
  - speech-translation
  - granite
  - granite-speech
  - autoregressive
  - quantized
  - 6-bit
---

# Granite Speech 4.1 2B MLX 6-bit

Quality-oriented MLX conversion of
[`ibm-granite/granite-speech-4.1-2b`](https://huggingface.co/ibm-granite/granite-speech-4.1-2b)
for Apple Silicon. This is the **autoregressive Granite Speech model**, not the
separate NAR variant.

The model runs with
[`mlx-audio`](https://github.com/Blaizzy/mlx-audio) and supports multilingual
automatic speech recognition (ASR), speech translation (AST), punctuation and
capitalization, and keyword-biased transcription.

## Quantization

This conversion uses post-training weight quantization only. No training,
fine-tuning, calibration dataset, or importance matrix was used.

| Component | Precision |
|---|---|
| 16-layer Conformer speech encoder | BF16 |
| 2-layer Q-Former speech projector | BF16 |
| Granite language model, eligible internal linear layers | MLX affine 6-bit, group size 64 |
| Token embedding | BF16 |
| Language-model output head | BF16 |
| Norms, biases, and unsupported tensors | BF16 |

The acoustic stack is kept in BF16 because quantization errors can compound
through the Conformer and alter the embeddings supplied to the language model.
The token embedding and output head are also kept in BF16 to preserve input and
output token precision. This follows the general multimodal quantization
practice of protecting encoders, projectors, embeddings, and output tensors
while compressing the large internal language-model projections.

- Source revision: `de575db64086f84fdc79da4932d1076e965bc546`
- Effective average reported by MLX: 10.144 bits per weight
- `model.safetensors`: approximately 2.7 GB

## Requirements

- Apple Silicon Mac
- macOS 14 or later
- `mlx-audio >= 0.4.5`

```bash
pip install -U mlx-audio
```

## Usage

```bash
python -m mlx_audio.stt.generate \
  --model divydeep/granite-speech-4.1-2b-mlx-6bit \
  --audio audio.wav \
  --output-path transcript \
  --format txt \
  --prompt "transcribe the speech with proper punctuation and capitalization."
```

Python:

```python
from mlx_audio.stt.utils import load_model

model = load_model("divydeep/granite-speech-4.1-2b-mlx-6bit")
result = model.generate(
    "audio.wav",
    prompt="transcribe the speech with proper punctuation and capitalization.",
    temperature=0.0,
)
print(result.text)
```

Greedy decoding with `temperature=0.0` is recommended for transcription.

## Prompts

| Task | Prompt |
|---|---|
| Raw ASR | `can you transcribe the speech into a written format?` |
| Punctuated ASR | `transcribe the speech with proper punctuation and capitalization.` |
| Keyword-biased ASR | `transcribe the speech to text. Keywords: <kw1>, <kw2>, ...` |
| Speech translation | `translate the speech to <language>.` |
| Punctuated translation | `translate the speech to <language> with proper punctuation and capitalization.` |

For non-English punctuation, translation, and keyword biasing, use the English
prompt forms recommended by IBM.

## Validation

The conversion was validated by:

- Strictly loading all weights with `mlx-audio`.
- Confirming the architecture is `granite_speech`, not `granite_speech_nar`.
- Running greedy transcription on IBM's bundled `multilingual_sample.wav`.
- Checking preservation of English and French text, punctuation, accents, and
  capitalization in the generated transcript.

This is a conversion smoke test, not a full WER benchmark. The upstream IBM
evaluation results describe the original model and should not be interpreted as
measured results for this quantized conversion.

## Limitations

- Quantization may affect rare words, names, punctuation, casing, keyword
  biasing, translation, and difficult or noisy audio.
- This model has not been independently evaluated on the complete IBM benchmark
  suite.
- The model is autoregressive and does not provide the throughput behavior of
  the separate NAR architecture.
- Refer to the upstream model card for intended use, language coverage, safety
  considerations, training data, and architectural details.

## References

- [Original model and authoritative model card](https://huggingface.co/ibm-granite/granite-speech-4.1-2b)
- [Granite Speech paper](https://arxiv.org/abs/2505.08699)
- [MLX Audio](https://github.com/Blaizzy/mlx-audio)

## License

Apache-2.0, matching the original model.

## Citation

```bibtex
@misc{granite-speech-4.1-2b,
  title={Granite 4.1 Speech},
  author={IBM Granite Speech Team},
  year={2026},
  url={https://huggingface.co/ibm-granite/granite-speech-4.1-2b}
}
```