---
library_name: mlx
license: apache-2.0
license_link: https://huggingface.co/Qwen/Qwen3.6-27B/blob/main/LICENSE
pipeline_tag: image-text-to-text
base_model: Qwen/Qwen3.6-27B
tags:
- mlx
- mtplx
- qwen3.6
- speculative-decoding
- multi-token-prediction
- 5-bit
---

# Qwen3.6 27B Q5 + MTPLX MTP

This artifact combines a uniform 5-bit MLX Qwen3.6 27B trunk with the
calibrated MTPLX MTP sidecar used by the optimized-speed release.

## Components

- Base model: `Qwen/Qwen3.6-27B`
- Q5 trunk: `mlx-community/Qwen3.6-27B-5bit`, revision
  `46f9b268ffe53528a8ad4ce8f684a480b3ef0d18`
- MTP sidecar: `Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed`, revision
  `be5190f2349594ec941753efc90a4ca5641af174`
- Trunk quantization: MLX affine 5-bit, group size 64
- MTP sidecar path: `mtp/weights.safetensors`
- MTPLX discovery metadata: `mtplx_runtime.json`

The Q5 trunk and optimized MTP sidecar are family-compatible but do not have a
published exactness contract for this combined artifact. MTPLX therefore marks
it `family-compatible-unverified` and requires explicit unsafe confirmation.

## Run with MTPLX

```bash
export MTPLX_VLLM_METAL_PAGED_TURBOQUANT=1
export MTPLX_VLLM_METAL_PAGED_TURBOQUANT_K_QUANT=q8_0
export MTPLX_VLLM_METAL_PAGED_TURBOQUANT_V_QUANT=q4_0
export MTPLX_PREFILL_CHUNK_SIZE=512
export MTPLX_PREFILL_CHUNK_SIZE_DENSE=512
export MTPLX_PREFILL_CHUNK_SIZE_REPAGE=512

mtplx serve \
  --model deanzchen/Qwen3.6-27B-Q5-MTPLX \
  --profile sustained \
  --generation-mode mtp \
  --depth 2 \
  --context-window 262144 \
  --prefill-chunk-tokens 512 \
  --temperature 0.6 \
  --top-p 0.95 \
  --top-k 20 \
  --draft-temperature 0.7 \
  --draft-top-p 0.95 \
  --draft-top-k 20 \
  --reasoning auto \
  --preserve-thinking scoped \
  --ssd-session-cache off \
  --no-stats-footer \
  --unsafe-force-unverified \
  --yes
```

TurboQuant normally activates after the sustained cache crosses 131,072
tokens. The configured 262,144-token maximum is a capacity target; this Q5
combination has been exercised to 132,515 tokens, not to the full maximum.

## Local M5 Max measurements

Hardware: M5 Max, 40 GPU cores, 48 GB unified memory. Runtime: MTPLX 2.0.2,
MLX 0.31.2, MLX-LM 0.31.3.

| Mode | Decode speed |
|---|---:|
| Autoregressive | 25.15 tok/s |
| MTP depth 1 | 41.29 tok/s |
| MTP depth 2 | 49.91 tok/s |
| MTP depth 3 | 44.95 tok/s |

Depth 2 was selected. The matched short run used approximately 22.1 GB peak
memory. A 132,515-token activation probe reached approximately 35.7 GB and
confirmed TurboQuant K8/V4, but initial decode at that extreme context was only
about 0.08 tok/s.

These are local measurements, not broad hardware claims.

The current CLI wrapper defers full verification-hidden materialization and
evaluates the committed hidden-state slice after target distributions. Five
token-identical D2 runs averaged 50.78 tok/s; the persisted wrapper's final
gate measured 49.82 tok/s with 100% acceptance.

For repeatable expectations and regression thresholds, see
`MTPLX_PERFORMANCE_BASELINE.json`. An 8-token smoke test is informational; use
the 128- or 256-token direct tuner workload as the performance gate.