Granite Speech 4.1 2B MLX Quantizations
Collection
Apple Silicon MLX conversions of IBM Granite Speech 4.1 2B, from memory-oriented 4-bit to quality-oriented 8-bit. Use with mlx-audio. • 4 items • Updated
Granite Speech 4.1 2B MLX 8-bit
Quality-oriented MLX conversion of
ibm-granite/granite-speech-4.1-2b
for Apple Silicon. This is the autoregressive Granite Speech model, not the NAR
variant.
Quantization
This conversion uses post-training weight quantization without training, calibration data, or an importance matrix.
| Component | Precision |
|---|---|
| 16-layer Conformer speech encoder | BF16 |
| 2-layer Q-Former speech projector | BF16 |
| Eligible internal language-model linear layers | MLX affine 8-bit, group size 64 |
| Token embedding and language-model output head | BF16 |
| Norms, biases, and unsupported tensors | BF16 |
- Source revision:
de575db64086f84fdc79da4932d1076e965bc546 - Effective average reported by MLX: 11.377 bits per weight
model.safetensors: approximately 3.14 GB
Usage
pip install -U mlx-audio
python -m mlx_audio.stt.generate \
--model /path/to/granite-speech-4.1-2b-mlx-8bit \
--audio audio.wav \
--output-path transcript \
--format txt \
--prompt "transcribe the speech with proper punctuation and capitalization."
Validation
The checkpoint was strictly loaded by mlx-audio and run with greedy decoding
on IBM's bundled multilingual_sample.wav. Its English and French transcript,
including punctuation, capitalization, accents, and hyphenation, exactly matched
the BF16 reference. This is a smoke test, not a complete WER benchmark.
Quantization may still affect names, rare words, translation, keyword biasing, and difficult or noisy audio.
License
Apache-2.0, matching the original model.
- Downloads last month
- 34
Model size
1B params
Tensor type
BF16
·
U32 ·
Hardware compatibility
Log In to add your hardware
8-bit
Model tree for divydeep/granite-speech-4.1-2b-mlx-8bit
Base model
ibm-granite/granite-4.0-1b-base Finetuned
ibm-granite/granite-speech-4.1-2b