EigenLabs/Qwen3.5-35B-A3B-MLX-VL-4bit-g64

Qwen3.5-35B-A3B MLX VLM Affine W4/g64 With Inline Affine W4 MTP

This is a full vision-language MLX conversion of Qwen/Qwen3.5-35B-A3B, created from exact source revision 59d61f3ce65a6d9863b86d2e96597125219dc754.

Every quantizable target and inline-MTP module uses affine 4-bit weights with group size 64. There are no 8-bit router overrides. As in the standard MLX-VLM conversion scope, the BF16 vision tower and normalization vectors remain unquantized.

The indexed safetensors artifact contains:

• The language target: 512 affine W4/g64 modules, including every MoE router.
• The complete BF16 vision tower: 333 tensors and 446,571,248 parameters.
• The complete native MTP subtree under mtp.*: 13 affine W4/g64 modules, including both MTP routers, plus BF16 norms.

The MTP head reuses the target embedding and LM head as defined by the source architecture.

Runtime requirements

Use mlx-vlm>=0.6.12 for ordinary text and image inference. This artifact was tested with mlx-vlm 0.6.15.

pip install -U "mlx-vlm>=0.6.12" jinja2

mlx_vlm.generate \
  --model EigenLabs/Qwen3.5-35B-A3B-MLX-VL-4bit-g64 \
  --prompt "Explain speculative decoding." \
  --max-tokens 256 \
  --temperature 0

Add --image /path/to/image.jpg for vision-language inference.

Current public mlx-vlm loads and serves the target while ignoring the namespaced mtp.* tensors. This repository does not include a Python Qwen MTP driver. The inline MTP payload is intended for runtimes that implement Qwen3.5 MTP, including Darkbloom's MLX-Swift engine.

Artifact

Property	Value
Indexed tensors	2,136
Target and vision tensors	2,090
Serialized MTP tensors	46
Source MTP tensors	785
Source MTP parameters	844,640,768
Quantized modules	525, all affine W4/g64
MTP payload	475,125,888 bytes
Combined tensor payload	20,866,531,040 bytes
Safetensors shards	5

The MLX converter reports 4.647 effective bits per target weight after scales, biases, and the unquantized vision tower are included.

Machine-readable provenance and verification evidence are in quantization_manifest.json, verification.json, and SHA256SUMS.

Local benchmark

Target-only greedy generation on an Apple M4 Max with mlx-vlm 0.6.15:

• Prompt: 1,701 tokens
• Generation: 128 tokens
• Warm-up: one discarded run
• Measurements: three runs
• Temperature: 0.0

Metric	Median	Mean	Range
Prefill	1,642.06 tok/s	1,642.60 tok/s	1,641.14–1,644.61 tok/s
Decode	113.67 tok/s	112.22 tok/s	109.23–113.75 tok/s
Peak memory	23.25 GB	—	—

These are local machine measurements, not cross-device guarantees. MTP was not active in this benchmark.

Verification scope

Verified:

• All 27 files in the pinned source snapshot passed Hugging Face cache verification.
• The converted index exactly covers every tensor in all five safetensors files.
• All 525 quantized modules resolve to affine W4/g64; no module has an 8-bit override.
• All 785 source MTP tensors are represented by the 46 serialized MLX tensors.
• Representative affine dequantization checks passed for fusion, attention, expert, and router weights.
• Qwen zero-centered RMSNorm conversion matched exactly.
• chat_template.jinja is byte-identical to the official source.
• A clean mlx-vlm 0.6.15 target-only inference and three-run throughput benchmark completed.

Not yet verified:

• Pixel-grounded image output for this exact conversion.
• Qwen MTP speculative token parity or speedup.
• Video inference.
• Population-level quality after quantization.

License

The base model is released under Apache 2.0. See the original Qwen/Qwen3.5-35B-A3B repository for its model card and license.