igorvibes/Qwen3.6-27B-UD-Q4_K_XL-AWQ-MTP-mlx

Qwen3.6-27B — UD-Q4_K_XL + AWQ + MTP (MLX)

Mixed-precision 4-bit quantization of Qwen/Qwen3.6-27B for Apple Silicon, built with mlx-node using the Unsloth Dynamic class map plus AWQ pre-scaling from an activation imatrix.

The multi-token-prediction (MTP) head is preserved. All 15 mtp.* tensors are kept inline in the main shards, unquantized in BF16, so a runtime that supports Qwen3.5/3.6 speculative decoding can use them.

This is the one deliberate difference from the Brooooooklyn UD-Q*_K_XL builds this recipe otherwise follows — I checked their Q4 checkpoint and it carries zero mtp.* tensors. Other MLX conversions of this model do ship MTP; this is not a claim to be the only one.

Contents

Component	Tensors	On disk	Precision
Language model	1,847	17.81 GiB	mixed 4–8 bit
Vision tower	333	0.86 GiB	BF16 (unquantized)
MTP head	15	0.79 GiB	BF16 (unquantized)
Total	2,195	19.45 GiB

affine quantization, group size 64, 370 per-tensor overrides. 27,781,427,952 total parameters.

Effective bits per weight

6.18 BPW — 20,889,186,434 bytes on disk over 27,020,391,152 logical parameters.

This is well above the nominal 4, and that is expected rather than a defect. --q-bits 4 sets the base class in the Unsloth Dynamic map, not the average. Only gate_proj and up_proj actually sit at 4-bit; attention projections, the GatedDeltaNet input projections, down_proj, the embeddings and lm_head are all promoted. The BF16 vision tower and BF16 MTP head add further weight that no quantizer touches. Any "Q4" MLX quant of this family that reports a similar size is doing the same thing.

Precision map

Module	Width	Scope
down_proj	5-bit	64 layers (0–63)
embed_tokens	6-bit	single tensor
in_proj_a	8-bit	48 layers (0–62)
in_proj_b	8-bit	48 layers (0–62)
in_proj_qkv	6-bit	48 layers (0–62)
in_proj_z	6-bit	48 layers (0–62)
k_proj	6-bit	16 layers (3–63)
lm_head	8-bit	single tensor
o_proj	8-bit	16 layers (3–63)
out_proj	8-bit	48 layers (0–62)
q_proj	6-bit	16 layers (3–63)
v_proj	6-bit	16 layers (3–63)
gate_proj, up_proj	4-bit (base)	all 64 layers

Left in BF16 throughout: all RMSNorms, q_norm/k_norm, the GatedDeltaNet A_log / conv1d / dt_bias state parameters, the entire vision tower, and the entire MTP head.

AWQ calibration

AWQ pre-scaling amplifies activation-important weight columns and folds the inverse into the preceding norm — an output-preserving reparametrization that moves quantization error onto channels that matter less. Applied here across four dependency groups: norm→gate/up, up-rows→down-cols, input_layernorm→q/k/v, and input_layernorm→GatedDeltaNet in_proj_*.

The importance matrix is Unsloth's own calibration for this exact model — imatrix_unsloth.gguf_file from unsloth/Qwen3.6-27B-GGUF, internally tagged unsloth_calibration_Qwen3.6-27B.txt, 992 tensors over 76 chunks. An imatrix from a different checkpoint would apply without error and silently degrade the result, so this provenance matters.

Reproducing

mlx convert \
  --input  <Qwen/Qwen3.6-27B snapshot> \
  --output Qwen3.6-27B-UD-Q4_K_XL-AWQ-MTP-mlx \
  --model-type qwen3_5 \
  --quantize \
  --q-bits 4 \
  --q-group-size 64 \
  --q-recipe unsloth \
  --imatrix-path imatrix_unsloth.gguf_file \
  --q-mtp off

--q-mtp off means "do not split the MTP head into a separate drafter directory" — the tensors stay inline and unquantized. Use --q-mtp split instead if you want a standalone drafter.

Conversion cost on a 36 GB M-series Mac: 99 s, peak 15.85 GiB process RSS, 22.7 GiB MLX allocator peak. Swap: 7.89 GiB paged out / 5.76 GiB paged in (vm_stat, sampled across the run).

This needs a build of mlx-node containing the bounded-memory conversion fix (PR #118); before it, the AWQ path materialized the whole BF16 checkpoint at once and would not complete in 36 GB.

Variants

Repo	Base width	Size	BPW
igorvibes/Qwen3.6-27B-UD-Q5_K_XL-AWQ-MTP-mlx	5-bit	22.95 GiB	7.38
This model	4-bit	19.45 GiB	6.18

Sources

• Base weights — Qwen/Qwen3.6-27B
• Importance matrix — unsloth/Qwen3.6-27B-GGUF
• Quantization strategy — Unsloth Dynamic
• Conversion tool — mlx-node
• Recipe reference — Brooooooklyn/Qwen3.6-27B-UD-Q4_K_XL-mlx, whose flag set this follows apart from MTP retention

Not tested

Stated plainly so you can weigh it:

• No benchmarks were run on this build. No perplexity, no task evals, no throughput numbers. Nothing here claims a quality or speed result.
• MTP speculative decoding is preserved, not verified. The tensors are present and correctly shaped; whether your runtime engages them is on your runtime.
• The vision tower is carried through unquantized but untested. No image or video input was exercised.
• License is stated as Apache-2.0 following the upstream Qwen3 convention; verify against the base repo if it matters to you.