EigenLabs/Qwen3.6-35B-A3B-MLX-4bit-g64-router8

Qwen3.6-35B-A3B MLX W4/g64 + W8 Routers

This is a text-only MLX conversion of Qwen/Qwen3.6-35B-A3B, created from the exact source revision 995ad96eacd98c81ed38be0c5b274b04031597b0 with MLX 0.31.2 and MLX-LM 0.31.3.

It is the stock MLX-LM affine baseline:

• 432 eligible modules use W4/g64 affine weights.
• 80 router modules use W8/g64 affine weights: the 40 expert routers and 40 shared-expert gates.
• Activations remain BF16 and quantized dot products accumulate in FP32.
• DeltaNet depthwise convolutions, norms, A_log, dt_bias, and other unsupported/natural parameters remain at their runtime precision.

This is a mixed 4/8-bit-class artifact, not strict uniform INT4. Scale and bias metadata are included in its physical bitrate.

Scope

This repository contains the text model only. Stock Python MLX-LM 0.31.3 removes the source checkpoint's vision tower and MTP subtree during conversion. Do not use this artifact for image/video input or MTP speculative decoding.

Property	Value
Logical text parameters	34,660,608,768
Quantized modules	512
W4/g64 modules	432
W8/g64 router modules	80
Tensor payload	19,508,787,456 bytes
Tensor payload	18.168974 GiB
Physical bitrate	4.502815 BPW
Safetensors shards	4

The machine-readable provenance, tensor coverage, and shard checksums are in quantization_manifest.json and SHA256SUMS.

Use With MLX-LM

pip install "mlx-lm==0.31.3"

from mlx_lm import generate, load

model, tokenizer = load(
    "EigenLabs/Qwen3.6-35B-A3B-MLX-4bit-g64-router8"
)

messages = [{"role": "user", "content": "Explain mixture-of-experts routing."}]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

response = generate(
    model,
    tokenizer,
    prompt=prompt,
    max_tokens=512,
    verbose=True,
)

Reproduction

Download the pinned BF16 source revision first, then convert the local snapshot:

hf download Qwen/Qwen3.6-35B-A3B \
  --revision 995ad96eacd98c81ed38be0c5b274b04031597b0 \
  --local-dir /path/to/Qwen3.6-35B-A3B-BF16

HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 \
mlx_lm.convert \
  --hf-path /path/to/Qwen3.6-35B-A3B-BF16 \
  --mlx-path /path/to/Qwen3.6-35B-A3B-MLX-4bit-g64-router8 \
  --quantize \
  --q-group-size 64 \
  --q-bits 4 \
  --q-mode affine \
  --dtype bfloat16

The Qwen model-specific stock predicate automatically promotes mlp.gate and shared_expert_gate to W8/g64. Inspect the emitted quantization table rather than assuming every matrix is W4.

Verification And Quality Boundary

Every safetensors header and index entry was cross-checked. The verified model contains 1,757 tensors, exactly 512 packed weight/scale/bias module triples, all 80 expected W8 router overrides, and no vision or MTP tensors. A local load and greedy structured-output smoke test returned the expected exact JSON:

{"sum":423,"check":"ok"}

A small deterministic BF16-teacher falsification pass used 2,481 chat-templated tokens, 43 full-vocabulary positions, all 99,240 layer-token routing decisions, and one continuous 1,012-token DeltaNet rollout. This stock artifact measured:

Metric	Result
Mean / median KL(BF16
p95 / maximum KL	1.689293 / 2.331561 nats
BF16 top-1 agreement	90.70%
Mean top-10 overlap	77.21%
Final DeltaNet state relative error	0.138974

These are targeted smoke measurements, not population-quality, perplexity, or task-accuracy estimates. No claim of negligible or sub-1% quality degradation is made. Evaluate long context, tools, code, reasoning, multilingual behavior, and your own workload before production use.

Why This Baseline Exists

This artifact is the smaller and faster control for future learned/native and custom-format quantization research. A strict stock W4/g32 candidate measured better in the small teacher pass but uses about 2.01 GiB more parameter storage. A future route/state-calibrated learned g64 artifact will target strict-g32 quality at this artifact's size and runtime.

Canonical Q4_K is the leading separate-format research direction because it showed stronger real-weight reconstruction at the same 4.5 BPW. It requires new MLX/Metal and Swift support and will use a distinct versioned artifact; this repository will remain the stock affine compatibility baseline.

License

The base model is released under Apache 2.0. See the Qwen/Qwen3.6-35B-A3B repository for its original model card and license.