majentik commited on
Commit
c89d95d
·
verified ·
1 Parent(s): f34174d

docs: audio smoke evidence + audio-tower note + family status

Browse files
Files changed (1) hide show
  1. README.md +24 -3
README.md CHANGED
@@ -16,7 +16,7 @@ tags:
16
 
17
  MLX 8bit (affine, 8-bit, group size 64) quantization of [`Qwen/Qwen3-ASR-0.6B-hf`](https://huggingface.co/Qwen/Qwen3-ASR-0.6B-hf) at upstream revision `6aa69c382e2b426eee1f5870d4c95859a74b6445`, produced directly from the BF16 safetensors with `pipelines.mlx_direct_quantize` (majek repo). ~0.9 GiB on disk.
18
 
19
- > **Runtime caveat:** this is a weights + config pack published ahead of runtime availability; runtime support pending upstream mlx-lm support for the `qwen3_asr` audio-text architecture (the Hugging Face runtime requires transformers 5.13+ — `qwen3_asr` merged after the 5.12.1 release). No local inference smoke was performed on these artifacts, so the quality of this variant is not verified.
20
 
21
  ## Why this variant
22
 
@@ -28,14 +28,35 @@ Affine 8-bit, group size 64 — the highest-fidelity tier, roughly half of BF16
28
  python -m pipelines.mlx_direct_quantize --model qwen3-asr-0.6b --base-dir /tmp/mlx-direct-release/qwen3-asr-0.6b/base --out-dir /tmp/mlx-direct-release/qwen3-asr-0.6b/8bit --bits 8 --mode affine --group-size 64
29
  ```
30
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
31
  ## Family
32
 
33
  All MLX variants of this model ship together:
34
 
35
- - [majentik/Qwen3-ASR-0.6B-MLX-4bit](https://huggingface.co/majentik/Qwen3-ASR-0.6B-MLX-4bit)
36
  - [majentik/Qwen3-ASR-0.6B-MLX-6bit](https://huggingface.co/majentik/Qwen3-ASR-0.6B-MLX-6bit)
37
  - `majentik/Qwen3-ASR-0.6B-MLX-8bit` (this repo)
38
- - [majentik/Qwen3-ASR-0.6B-MLX-MXFP4](https://huggingface.co/majentik/Qwen3-ASR-0.6B-MLX-MXFP4)
39
 
40
  ## Provenance
41
 
 
16
 
17
  MLX 8bit (affine, 8-bit, group size 64) quantization of [`Qwen/Qwen3-ASR-0.6B-hf`](https://huggingface.co/Qwen/Qwen3-ASR-0.6B-hf) at upstream revision `6aa69c382e2b426eee1f5870d4c95859a74b6445`, produced directly from the BF16 safetensors with `pipelines.mlx_direct_quantize` (majek repo). ~0.9 GiB on disk.
18
 
19
+ > **Runtime caveat:** this is a weights + config pack published ahead of runtime availability; runtime support pending upstream mlx-lm support for the `qwen3_asr` audio-text architecture (the Hugging Face runtime requires transformers 5.13+ — `qwen3_asr` merged after the 5.12.1 release). This variant **passed** the local MLX audio-inference smoke gate — see *Audio inference smoke* below (run natively with the source repo's `pipelines/qwen3_asr_mlx` MLX harness).
20
 
21
  ## Why this variant
22
 
 
28
  python -m pipelines.mlx_direct_quantize --model qwen3-asr-0.6b --base-dir /tmp/mlx-direct-release/qwen3-asr-0.6b/base --out-dir /tmp/mlx-direct-release/qwen3-asr-0.6b/8bit --bits 8 --mode affine --group-size 64
29
  ```
30
 
31
+ ## Audio tower
32
+
33
+ The `model.audio_tower.*` windowed-attention audio encoder (297 tensors) and the `model.multi_modal_projector.*` projector (4 tensors) are passed through unquantized in BF16 — only the `model.language_model.*` qwen3 text decoder is quantized.
34
+
35
+ ## Audio inference smoke
36
+
37
+ **PASS** — this variant passed the audio-inference smoke gate on 2026-07-03.
38
+
39
+ Gate design: character error rate (CER) is measured against an **MLX-BF16 self-reference** baseline (the same MLX audio path run at BF16). Only the clips that are stable and in-language for both family sizes (`1_cantonese.wav`, `4_codeswitch_singlish.wav` — Cantonese and Singlish) decide the verdict; the other clips are transcribed and recorded as informational. Threshold: CER ≤ 0.05 on every gating clip.
40
+
41
+ | clip | CER vs MLX-BF16 | gating | wall_s |
42
+ | --- | --- | --- | --- |
43
+ | `1_cantonese.wav` | 0.0 | yes | 0.157 |
44
+ | `2_hokkien.wav` | 0.988701 | no | 0.289 |
45
+ | `3_codeswitch_tamil.wav` | 0.0 | no | 0.141 |
46
+ | `4_codeswitch_singlish.wav` | 0.0 | yes | 0.066 |
47
+
48
+ - Decoder-only smoke (one-token forward, finite logits): **pass**
49
+ - Total wall clock: 39.753 s
50
+ - Smoke run 2026-07-03 on Apple Silicon via the source repo's `pipelines/qwen3_asr_mlx` package (driver: `pipelines/qwen3_asr_mlx_smoke.py`).
51
+
52
  ## Family
53
 
54
  All MLX variants of this model ship together:
55
 
56
+ - `majentik/Qwen3-ASR-0.6B-MLX-4bit` (private — pending requantization; failed the audio-inference smoke gate)
57
  - [majentik/Qwen3-ASR-0.6B-MLX-6bit](https://huggingface.co/majentik/Qwen3-ASR-0.6B-MLX-6bit)
58
  - `majentik/Qwen3-ASR-0.6B-MLX-8bit` (this repo)
59
+ - `majentik/Qwen3-ASR-0.6B-MLX-MXFP4` (private — pending requantization; failed the audio-inference smoke gate)
60
 
61
  ## Provenance
62