EigenLabs/Qwen3.6-35B-A3B-MLX-VL-4bit-g64-router8

Qwen3.6-35B-A3B MLX VLM W4/g64 + W8 Routers + Inline MXFP8 MTP

This is a full vision-language MLX conversion of Qwen/Qwen3.6-35B-A3B, created from exact source revision 995ad96eacd98c81ed38be0c5b274b04031597b0.

The single indexed safetensors weight set contains all three components:

• The language target: 432 W4/g64 affine modules and 80 W8/g64 router modules.
• The complete 27-layer, 446,571,248-parameter BF16 vision tower.
• The complete native MTP subtree under mtp.*: 835,715,072 matrix parameters in MXFP8/g32 and 8,925,696 sensitive fusion/router/gate/norm parameters in BF16.

There is no mtp/ directory, separate MTP repository, or mtp.safetensors. The MTP head reuses the target embedding and LM head, as intended by the source architecture.

Runtime Requirements

Use mlx-vlm>=0.6.12 for normal text and image inference. Earlier Qwen3.5-MoE VLM loaders could reapply the source RMSNorm conversion to serialized MLX weights.

pip install -U "mlx-vlm>=0.6.12"

Normal mlx_vlm.load() ignores the namespaced mtp.* tensors safely. The included mtp_exact_generate.py constructs the drafter in memory directly from those entries in the same index. It does not use --draft-model, create a temporary weight copy, or download another repository.

Exact Qwen MoE Verification

Stock MLXVLM 0.6.12 uses a batched target-verification linear path whose greedy arithmetic can differ from ordinary one-token decoding for this mixed W4/W8 MoE target. Exact output parity requires target-verification linear projections to run one time position at a time. The artifact verification used that minimal runtime correction while leaving MoE expert gathers batched. Without it, a rejected proposal can change later output; do not claim exact speculative decoding from unpatched 0.6.12.

Artifact

Property	Value
Indexed tensors	2,120
Pre-existing VLM tensors	2,090
Serialized MTP tensors	30
Source MTP parameters	844,640,768
MXFP8/g32 MTP parameters	835,715,072
BF16 MTP parameters	8,925,696
MTP payload	879,682,560 bytes (0.819268 GiB)
Combined tensor payload	21,281,612,512 bytes (19.820046 GiB)
Safetensors shards	4

The machine-readable provenance, coverage, validations, and hashes are in quantization_manifest.json, verification.json, and SHA256SUMS.

Normal Inference

python -m mlx_vlm generate \
  --model EigenLabs/Qwen3.6-35B-A3B-MLX-VL-4bit-g64-router8 \
  --image /path/to/image.jpg \
  --prompt "Describe this image." \
  --max-tokens 256 \
  --temperature 0 \
  --thinking-mode disabled

Strict eager loading from the combined artifact passed:

• Pixel-grounded OCR/shape/color JSON: ORCHID-7319, count 27, target CYAN TRIANGLE, and the correct ordered shapes/colors.
• Natural photograph JSON: two cats, pink surface, and two remote controls.
• Text regression: {"sum":423,"check":"ok"}.

All ordinary outputs match the pre-MTP VLM artifact because every original VLM tensor is value-identical.

Speculative Decoding

python mtp_exact_generate.py \
  --model EigenLabs/Qwen3.6-35B-A3B-MLX-VL-4bit-g64-router8 \
  --draft-block-size 3 \
  --prompt "Explain speculative decoding." \
  --max-tokens 256 \
  --temperature 0

The final parity suite used five workloads, three target-only runs and three MTP runs per workload, for 30 paired runs total. It included structured text, technical prose, code, a length-limited sequence, and image prefill.

Metric	Result
Exact token parity	15/15 MTP runs
Exact text parity	15/15 MTP runs
Drafted tokens	1,671
Accepted draft tokens	1,353
Rejected draft tokens	318
Aggregate acceptance	80.97%
Median per-case speedup	1.284x
Per-case speedup range	1.112x to 1.518x

Every case was faster than target-only decoding. The 318 rejections exercised target cache rollback repeatedly, including three rejections after image prefill. These are local M4 Max measurements, not cross-device guarantees.

Structural Verification

• All 2,090 pre-existing VLM tensors are value-identical to the prior published revision.
• All 333 vision tensors remain BF16 and value-exact with the pinned source after the required patch-convolution transpose.
• All 19 source MTP tensors and 844,640,768 source parameters are represented.
• The 30 serialized MTP tensors were independently rebuilt from the pinned source and match exactly.
• The MTP expert, shared-expert, and attention matrices are MXFP8/g32; fusion fc, routers, gates, and norms remain BF16.
• The combined index covers all 2,120 tensors exactly and reports the exact 21,281,612,512-byte payload.

Scope

Image and text inference plus greedy MTP parity are verified. Video processor files are preserved, but video inference was not tested. The validation suite is targeted evidence, not a broad multimodal or population-quality benchmark.

License

The base model is released under Apache 2.0. See the original Qwen/Qwen3.6-35B-A3B repository for its model card and license.