---
license: apache-2.0
library_name: mlx
pipeline_tag: automatic-speech-recognition
base_model: ibm-granite/granite-speech-4.1-2b
base_model_relation: quantized
language:
  - multilingual
  - en
  - fr
  - de
  - es
  - pt
  - ja
tags:
  - mlx
  - mlx-audio
  - automatic-speech-recognition
  - speech-to-text
  - speech-translation
  - granite
  - granite-speech
  - autoregressive
  - quantized
  - 4-bit
---

# Granite Speech 4.1 2B MLX 4-bit

Memory-oriented MLX conversion of
[`ibm-granite/granite-speech-4.1-2b`](https://huggingface.co/ibm-granite/granite-speech-4.1-2b)
for Apple Silicon. This is the autoregressive Granite Speech model, not the NAR
variant.

## Quantization

This conversion uses post-training weight quantization without training,
calibration data, or an importance matrix.

| Component | Precision |
|---|---|
| 16-layer Conformer speech encoder | BF16 |
| 2-layer Q-Former speech projector | BF16 |
| Eligible internal language-model linear layers | MLX affine 4-bit, group size 64 |
| Token embedding and language-model output head | BF16 |
| Norms, biases, and unsupported tensors | BF16 |

- Source revision: `de575db64086f84fdc79da4932d1076e965bc546`
- Effective average reported by MLX: 8.911 bits per weight
- `model.safetensors`: approximately 2.46 GB

## Usage

```bash
pip install -U mlx-audio

python -m mlx_audio.stt.generate \
  --model /path/to/granite-speech-4.1-2b-mlx-4bit \
  --audio audio.wav \
  --output-path transcript \
  --format txt \
  --prompt "transcribe the speech with proper punctuation and capitalization."
```

## Validation

The checkpoint was strictly loaded by `mlx-audio` and run with greedy decoding
on IBM's bundled `multilingual_sample.wav`. It preserved every reference word
and accent, but omitted three French punctuation or hyphen marks compared with
the BF16 reference. This is a smoke test, not a complete WER benchmark.

Quantization can affect names, rare words, punctuation, casing, translation,
keyword biasing, and difficult or noisy audio. Use the 8-bit variant when closer
agreement with the original model is more important than memory usage.

## License

Apache-2.0, matching the original model.