Vontra/GLM-5.3-Flash-MLX-2bit-MTP

GLM-5.3-Flash, hybrid MLX 2-bit with native MTP

A deterministic quality-protected MLX conversion of zai-org/GLM-5.3-Flash, retaining the model's matching native next-token prediction layer.

Original model · Z.ai overview · Technical report · Apple MLX · MIT licence

This is a hybrid 2-bit checkpoint, not a uniform Q2 build. Routed trunk experts use Q2, non-expert language projections use Q8, native-MTP expert projections use Q4, and sensitive components remain BF16. This allocation passed live generation where broader Q2 recipes did not.

At a glance

Item	Value
Repository	Vontra/GLM-5.3-Flash-MLX-2bit-MTP
Base model	zai-org/GLM-5.3-Flash
Source revision	04c4e9e95c5da8862dced7e5056455116f83a7e0
Source weight format	FP8 E4M3 with 128x128 block scaling
Output format	MLX safetensors, affine weight quantisation, group size 64
Quantisation	Deterministic hybrid 2-bit recipe
Native MTP	Preserved, one upstream prediction layer, runtime depth 1
Indexed tensors	114,154
Weight shards	26
Tensor payload	111,317,859,192 bytes, 111.318 GB / 103.673 GiB
Configured context	1,048,576 tokens
Architecture	glm5_next, multimodal sparse MoE

Precision recipe

Component	Treatment
Routed trunk experts	Q2 affine, group size 64; 36,288 source matrices stacked into 126 runtime modules
Non-expert language projections	Q8 affine, group size 64; 540 source matrices mapped to 552 runtime modules
Native-MTP routed experts	Q4 affine, group size 64; 864 source matrices stacked into 3 runtime modules
Sparse indexer projections	All 36 at Q8 affine, group size 64
Token embedding and output head	BF16
Vision encoder and projector	BF16
MTP fusion projection	BF16
Routers, hyper-connections, norms and other non-quantisable tensors	Source precision

This is weight-only post-training quantisation. It does not retrain or fine-tune the upstream model.

Runtime compatibility

GLM-5.3-Flash uses the glm5_next multimodal architecture and a native NextN/MTP block. Use a runtime that supports this architecture and per-module MLX quantisation metadata.

Component	Tested version
oMLX	0.6.3rc3, build 2475
MLX	0.32.0
mlx-lm	0.31.3
mlx-vlm	0.6.3
Native-MTP draft depth	1

The current upstream chat template defaults to maximum reasoning effort. For short, direct answers, pass reasoning_effort: low through the chat-template arguments.

Download and use

hf download Vontra/GLM-5.3-Flash-MLX-2bit-MTP \
  --local-dir ./GLM-5.3-Flash-MLX-2bit-MTP

Add the downloaded directory to a compatible oMLX model directory, refresh the model registry, and select the model. Native MTP is optional and uses draft depth 1.

For an OpenAI-compatible request through oMLX, include the upstream low-reasoning template option when you want a concise answer:

{
  "model": "GLM-5.3-Flash-MLX-2bit-MTP",
  "messages": [{"role": "user", "content": "What is the capital of France?"}],
  "chat_template_kwargs": {"reasoning_effort": "low"}
}

Apple M3 Studio performance

Each result is the median of three 512-token text-generation runs after a separate warm-up. Both modes used the same checkpoint, prompt, deterministic sampling settings, and current upstream chat template.

Runtime mode	Runs	Output per run	Median decode speed	Long-run output parity
Native MTP disabled	3	512 tokens	6.063 tok/s	Reference
Native MTP enabled, depth 1	3	512 tokens	6.257 tok/s	Exact match

Native MTP improved median decode throughput by 3.21% in this test. All six 512-token runs produced the same output hash. MTP gains depend on the prompt and draft acceptance, so treat this as a practical reference for the tested Studio rather than a universal result.

Architecture

GLM-5.3-Flash combines KDA linear-attention layers with periodic sparse-attention layers, a sparse mixture-of-experts feed-forward stack, manifold-constrained hyper-connections, a vision encoder, and one native next-token prediction layer.

Architecture detail	Upstream value
Parameters	320B total / 18B active
Language layers	45
Routed / active experts	288 / 8, plus 1 shared expert
Hidden size	4,096
Attention heads	64
Native MTP layers	1
Configured maximum context	1,048,576 tokens

See the official model card, Z.ai overview, and GLM-5 technical report for upstream training, evaluations, intended uses, and safety guidance.

Validation

Check	Result
Official source structure	76,108 tensors, 62 shards and 37,338 FP8 weight-scale pairs validated before conversion
Safetensors index and shard resolution	114,154 entries resolve to 26 final shards
Saved precision layout	36,288 Q2, 864 Q4 and 540 Q8 source matrices; all quantised matrices have matching weight, scale and bias tensors
Vision payload	347 BF16 source tensors preserved
Native MTP structure	Complete upstream prediction layer preserved; runtime reports native-MTP compatibility
MTP disabled generation	Deterministic factual, arithmetic, instruction and coherence checks passed
MTP enabled generation	The same checks passed with exact output parity
Sustained generation	Three 512-token runs per mode; 6.063 tok/s off and 6.257 tok/s on

Limitations

• Hybrid quantisation can reduce quality relative to the official checkpoint. The effect depends on the workload.
• The configured one-million-token context does not mean every Apple silicon system has enough memory for a full-context request.
• Image and video prefill have different memory and throughput characteristics from text-only generation. The published throughput numbers are text-only.
• Native MTP may be neutral or slower on prompts with low draft acceptance.
• Runtime support for glm5_next, mixed per-module quantisation, and native MTP is version-sensitive.

This is a community quantisation, not an official Z.ai release.

Licence and attribution

The upstream model uses the MIT License. The official licence text is included as LICENSE.

Model design, training, upstream evaluations, and documentation belong to Z.ai and the GLM-5 contributors. The MLX conversion, native-MTP preservation, validation, and packaging are provided by Vontra.

If you use this model in research, cite the upstream report:

@misc{glm5team2026glm5,
  title        = {GLM-5: from Vibe Coding to Agentic Engineering},
  author       = {GLM-5-Team and others},
  year         = {2026},
  eprint       = {2602.15763},
  archivePrefix= {arXiv},
  primaryClass = {cs.LG},
  url          = {https://arxiv.org/abs/2602.15763}
}