Instructions to use majentik/Qwen3-ASR-0.6B-MLX-8bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use majentik/Qwen3-ASR-0.6B-MLX-8bit with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] huggingface-cli download --local-dir Qwen3-ASR-0.6B-MLX-8bit majentik/Qwen3-ASR-0.6B-MLX-8bit
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
docs: audio smoke evidence + audio-tower note + family status
Browse files
README.md
CHANGED
|
@@ -16,7 +16,7 @@ tags:
|
|
| 16 |
|
| 17 |
MLX 8bit (affine, 8-bit, group size 64) quantization of [`Qwen/Qwen3-ASR-0.6B-hf`](https://huggingface.co/Qwen/Qwen3-ASR-0.6B-hf) at upstream revision `6aa69c382e2b426eee1f5870d4c95859a74b6445`, produced directly from the BF16 safetensors with `pipelines.mlx_direct_quantize` (majek repo). ~0.9 GiB on disk.
|
| 18 |
|
| 19 |
-
> **Runtime caveat:** this is a weights + config pack published ahead of runtime availability; runtime support pending upstream mlx-lm support for the `qwen3_asr` audio-text architecture (the Hugging Face runtime requires transformers 5.13+ — `qwen3_asr` merged after the 5.12.1 release).
|
| 20 |
|
| 21 |
## Why this variant
|
| 22 |
|
|
@@ -28,14 +28,35 @@ Affine 8-bit, group size 64 — the highest-fidelity tier, roughly half of BF16
|
|
| 28 |
python -m pipelines.mlx_direct_quantize --model qwen3-asr-0.6b --base-dir /tmp/mlx-direct-release/qwen3-asr-0.6b/base --out-dir /tmp/mlx-direct-release/qwen3-asr-0.6b/8bit --bits 8 --mode affine --group-size 64
|
| 29 |
```
|
| 30 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 31 |
## Family
|
| 32 |
|
| 33 |
All MLX variants of this model ship together:
|
| 34 |
|
| 35 |
-
-
|
| 36 |
- [majentik/Qwen3-ASR-0.6B-MLX-6bit](https://huggingface.co/majentik/Qwen3-ASR-0.6B-MLX-6bit)
|
| 37 |
- `majentik/Qwen3-ASR-0.6B-MLX-8bit` (this repo)
|
| 38 |
-
-
|
| 39 |
|
| 40 |
## Provenance
|
| 41 |
|
|
|
|
| 16 |
|
| 17 |
MLX 8bit (affine, 8-bit, group size 64) quantization of [`Qwen/Qwen3-ASR-0.6B-hf`](https://huggingface.co/Qwen/Qwen3-ASR-0.6B-hf) at upstream revision `6aa69c382e2b426eee1f5870d4c95859a74b6445`, produced directly from the BF16 safetensors with `pipelines.mlx_direct_quantize` (majek repo). ~0.9 GiB on disk.
|
| 18 |
|
| 19 |
+
> **Runtime caveat:** this is a weights + config pack published ahead of runtime availability; runtime support pending upstream mlx-lm support for the `qwen3_asr` audio-text architecture (the Hugging Face runtime requires transformers 5.13+ — `qwen3_asr` merged after the 5.12.1 release). This variant **passed** the local MLX audio-inference smoke gate — see *Audio inference smoke* below (run natively with the source repo's `pipelines/qwen3_asr_mlx` MLX harness).
|
| 20 |
|
| 21 |
## Why this variant
|
| 22 |
|
|
|
|
| 28 |
python -m pipelines.mlx_direct_quantize --model qwen3-asr-0.6b --base-dir /tmp/mlx-direct-release/qwen3-asr-0.6b/base --out-dir /tmp/mlx-direct-release/qwen3-asr-0.6b/8bit --bits 8 --mode affine --group-size 64
|
| 29 |
```
|
| 30 |
|
| 31 |
+
## Audio tower
|
| 32 |
+
|
| 33 |
+
The `model.audio_tower.*` windowed-attention audio encoder (297 tensors) and the `model.multi_modal_projector.*` projector (4 tensors) are passed through unquantized in BF16 — only the `model.language_model.*` qwen3 text decoder is quantized.
|
| 34 |
+
|
| 35 |
+
## Audio inference smoke
|
| 36 |
+
|
| 37 |
+
**PASS** — this variant passed the audio-inference smoke gate on 2026-07-03.
|
| 38 |
+
|
| 39 |
+
Gate design: character error rate (CER) is measured against an **MLX-BF16 self-reference** baseline (the same MLX audio path run at BF16). Only the clips that are stable and in-language for both family sizes (`1_cantonese.wav`, `4_codeswitch_singlish.wav` — Cantonese and Singlish) decide the verdict; the other clips are transcribed and recorded as informational. Threshold: CER ≤ 0.05 on every gating clip.
|
| 40 |
+
|
| 41 |
+
| clip | CER vs MLX-BF16 | gating | wall_s |
|
| 42 |
+
| --- | --- | --- | --- |
|
| 43 |
+
| `1_cantonese.wav` | 0.0 | yes | 0.157 |
|
| 44 |
+
| `2_hokkien.wav` | 0.988701 | no | 0.289 |
|
| 45 |
+
| `3_codeswitch_tamil.wav` | 0.0 | no | 0.141 |
|
| 46 |
+
| `4_codeswitch_singlish.wav` | 0.0 | yes | 0.066 |
|
| 47 |
+
|
| 48 |
+
- Decoder-only smoke (one-token forward, finite logits): **pass**
|
| 49 |
+
- Total wall clock: 39.753 s
|
| 50 |
+
- Smoke run 2026-07-03 on Apple Silicon via the source repo's `pipelines/qwen3_asr_mlx` package (driver: `pipelines/qwen3_asr_mlx_smoke.py`).
|
| 51 |
+
|
| 52 |
## Family
|
| 53 |
|
| 54 |
All MLX variants of this model ship together:
|
| 55 |
|
| 56 |
+
- `majentik/Qwen3-ASR-0.6B-MLX-4bit` (private — pending requantization; failed the audio-inference smoke gate)
|
| 57 |
- [majentik/Qwen3-ASR-0.6B-MLX-6bit](https://huggingface.co/majentik/Qwen3-ASR-0.6B-MLX-6bit)
|
| 58 |
- `majentik/Qwen3-ASR-0.6B-MLX-8bit` (this repo)
|
| 59 |
+
- `majentik/Qwen3-ASR-0.6B-MLX-MXFP4` (private — pending requantization; failed the audio-inference smoke gate)
|
| 60 |
|
| 61 |
## Provenance
|
| 62 |
|