README.md

---
license: apache-2.0
language:
  - en
  - zh
base_model: Qwen/Qwen3.6-27B
tags:
  - mlx
  - mlx-node
  - quantized
  - awq
  - mtp
  - 4-bit
  - qwen3.6
  - hybrid-attention
  - gated-delta-net
  - apple-silicon
  - unsloth-dynamic
library_name: mlx-node
quantized_by: igorvibes
pipeline_tag: text-generation
model_type: qwen3_5
---

# Qwen3.6-27B — UD-Q4_K_XL + AWQ + MTP (MLX)

Mixed-precision 4-bit quantization of [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B)
for Apple Silicon, built with [mlx-node](https://github.com/mlx-node/mlx-node) using the
**Unsloth Dynamic** class map plus **AWQ pre-scaling** from an activation imatrix.

**The multi-token-prediction (MTP) head is preserved.** All 15 `mtp.*` tensors are
kept inline in the main shards, unquantized in BF16, so a runtime that supports
Qwen3.5/3.6 speculative decoding can use them.

This is the one deliberate difference from the
[Brooooooklyn UD-Q*_K_XL](https://huggingface.co/Brooooooklyn/Qwen3.6-27B-UD-Q4_K_XL-mlx)
builds this recipe otherwise follows — I checked their Q4 checkpoint and it carries
zero `mtp.*` tensors. Other MLX conversions of this model do ship MTP; this is not a
claim to be the only one.

## Contents

| Component | Tensors | On disk | Precision |
|---|---:|---:|---|
| Language model | 1,847 | 17.81 GiB | mixed 4–8 bit |
| Vision tower | 333 | 0.86 GiB | BF16 (unquantized) |
| MTP head | 15 | 0.79 GiB | BF16 (unquantized) |
| **Total** | **2,195** | **19.45 GiB** | |

affine quantization, group size 64,
370 per-tensor overrides.
27,781,427,952 total parameters.

## Effective bits per weight

**6.18 BPW** — 20,889,186,434 bytes on disk over 27,020,391,152 logical parameters.

This is well above the nominal 4, and that is expected rather than a defect.
`--q-bits 4` sets the *base* class in the Unsloth Dynamic map, not the average.
Only `gate_proj` and `up_proj` actually sit at 4-bit; attention projections,
the GatedDeltaNet input projections, `down_proj`, the embeddings and `lm_head` are
all promoted. The BF16 vision tower and BF16 MTP head add further weight that no
quantizer touches. Any "Q4" MLX quant of this family that reports a similar size
is doing the same thing.

## Precision map

| Module | Width | Scope |
|---|---|---|
| `down_proj` | 5-bit | 64 layers (0–63) |
| `embed_tokens` | 6-bit | single tensor |
| `in_proj_a` | 8-bit | 48 layers (0–62) |
| `in_proj_b` | 8-bit | 48 layers (0–62) |
| `in_proj_qkv` | 6-bit | 48 layers (0–62) |
| `in_proj_z` | 6-bit | 48 layers (0–62) |
| `k_proj` | 6-bit | 16 layers (3–63) |
| `lm_head` | 8-bit | single tensor |
| `o_proj` | 8-bit | 16 layers (3–63) |
| `out_proj` | 8-bit | 48 layers (0–62) |
| `q_proj` | 6-bit | 16 layers (3–63) |
| `v_proj` | 6-bit | 16 layers (3–63) |
| `gate_proj`, `up_proj` | **4-bit** (base) | all 64 layers |

Left in BF16 throughout: all RMSNorms, `q_norm`/`k_norm`, the GatedDeltaNet
`A_log` / `conv1d` / `dt_bias` state parameters, the entire vision tower, and the
entire MTP head.

## AWQ calibration

AWQ pre-scaling amplifies activation-important weight columns and folds the
inverse into the preceding norm — an output-preserving reparametrization that
moves quantization error onto channels that matter less. Applied here across
four dependency groups: norm→gate/up, up-rows→down-cols, input_layernorm→q/k/v,
and input_layernorm→GatedDeltaNet `in_proj_*`.

The importance matrix is Unsloth's own calibration for **this exact model** —
`imatrix_unsloth.gguf_file` from
[unsloth/Qwen3.6-27B-GGUF](https://huggingface.co/unsloth/Qwen3.6-27B-GGUF),
internally tagged `unsloth_calibration_Qwen3.6-27B.txt`, 992 tensors over 76 chunks.
An imatrix from a different checkpoint would apply without error and silently
degrade the result, so this provenance matters.

## Reproducing

```bash
mlx convert \
  --input  <Qwen/Qwen3.6-27B snapshot> \
  --output Qwen3.6-27B-UD-Q4_K_XL-AWQ-MTP-mlx \
  --model-type qwen3_5 \
  --quantize \
  --q-bits 4 \
  --q-group-size 64 \
  --q-recipe unsloth \
  --imatrix-path imatrix_unsloth.gguf_file \
  --q-mtp off
```

`--q-mtp off` means "do not split the MTP head into a separate drafter directory" —
the tensors stay inline and unquantized. Use `--q-mtp split` instead if you want a
standalone drafter.

Conversion cost on a 36 GB M-series Mac: **99 s**, peak
15.85 GiB process RSS, 22.7 GiB MLX allocator peak.
Swap: 7.89 GiB paged out / 5.76 GiB paged in (`vm_stat`, sampled across the run).

This needs a build of mlx-node containing the bounded-memory conversion fix
([PR #118](https://github.com/mlx-node/mlx-node/pull/118)); before it, the AWQ path
materialized the whole BF16 checkpoint at once and would not complete in 36 GB.

## Variants

| Repo | Base width | Size | BPW |
|---|---|---:|---:|
| [igorvibes/Qwen3.6-27B-UD-Q5_K_XL-AWQ-MTP-mlx](https://huggingface.co/igorvibes/Qwen3.6-27B-UD-Q5_K_XL-AWQ-MTP-mlx) | 5-bit | 22.95 GiB | 7.38 |
| **This model** | **4-bit** | **19.45 GiB** | **6.18** |

## Sources

- Base weights — [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B)
- Importance matrix — [unsloth/Qwen3.6-27B-GGUF](https://huggingface.co/unsloth/Qwen3.6-27B-GGUF)
- Quantization strategy — [Unsloth Dynamic](https://unsloth.ai/docs/models/qwen3.5/gguf-benchmarks)
- Conversion tool — [mlx-node](https://github.com/mlx-node/mlx-node)
- Recipe reference — [Brooooooklyn/Qwen3.6-27B-UD-Q4_K_XL-mlx](https://huggingface.co/Brooooooklyn/Qwen3.6-27B-UD-Q4_K_XL-mlx),
  whose flag set this follows apart from MTP retention

## Not tested

Stated plainly so you can weigh it:

- **No benchmarks were run on this build.** No perplexity, no task evals, no
  throughput numbers. Nothing here claims a quality or speed result.
- **MTP speculative decoding is preserved, not verified.** The tensors are present
  and correctly shaped; whether your runtime engages them is on your runtime.
- **The vision tower is carried through unquantized but untested.** No image or
  video input was exercised.
- License is stated as Apache-2.0 following the upstream Qwen3 convention; verify
  against the base repo if it matters to you.